Jul 30 09:49:21.867697 ip-10-0-128-146 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 09:49:22.246020 ip-10-0-128-146 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 09:49:22.246020 ip-10-0-128-146 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 09:49:22.246020 ip-10-0-128-146 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 09:49:22.246020 ip-10-0-128-146 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 09:49:22.246020 ip-10-0-128-146 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 09:49:22.247004 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.246882    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 09:49:22.251858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251837    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 09:49:22.251858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251857    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251861    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251864    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251868    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251872    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251874    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251877    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251880    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251883    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251886    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251889    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251892    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251894    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251897    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251901    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251904    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251906    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251909    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251911    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251914    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 09:49:22.251932 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251917    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251926    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251929    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251932    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251934    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251937    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251941    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251943    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251946    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251949    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251951    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251954    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251956    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251959    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251962    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251964    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251967    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251969    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251972    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 09:49:22.252431 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251974    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251977    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251979    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251981    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251984    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251986    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251989    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251992    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251996    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.251998    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252000    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252003    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252005    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252009    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252011    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252014    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252020    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252024    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252027    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 09:49:22.252894 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252030    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252033    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252036    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252039    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252042    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252045    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252048    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252051    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252054    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252057    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252059    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252062    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252064    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252067    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252069    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252072    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252074    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252077    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252079    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252082    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 09:49:22.253375 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252085    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252088    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252092    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252096    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252101    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252104    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252107    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252109    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252113    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252116    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252118    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252121    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252123    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252126    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252129    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252131    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252133    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252136    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252138    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 09:49:22.253851 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252141    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252885    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252893    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252896    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252899    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252902    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252905    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252908    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252911    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252913    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252916    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252919    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252921    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252924    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252927    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252930    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252932    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252935    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252938    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252940    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 09:49:22.254348 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252943    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252947    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252949    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252952    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252955    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252957    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252960    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252963    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252965    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252968    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252970    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252973    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252977    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252981    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252984    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252986    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252989    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252991    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252994    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252997    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 09:49:22.254859 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.252999    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253002    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253004    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253007    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253009    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253012    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253014    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253017    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253021    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253023    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253026    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253028    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253032    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253035    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253038    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253040    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253043    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253045    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253048    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253051    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 09:49:22.255371 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253053    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253056    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253058    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253061    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253063    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253066    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253068    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253071    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253074    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253076    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253078    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253081    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253083    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253086    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253088    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253091    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253093    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253096    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253098    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253103    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 09:49:22.255879 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253107    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253110    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253114    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253117    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253121    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253124    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253127    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253131    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253133    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253136    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253139    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253142    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253145    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253148    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253150    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253153    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253156    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253159    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253162    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 09:49:22.256383 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253165    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253627    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253633    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253637    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253640    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253643    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253645    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253648    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253650    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253653    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253656    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253658    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253662    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253666    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253672    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253674    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253677    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253680    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253684    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 09:49:22.256849 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253688    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253692    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253694    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253697    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253700    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253703    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253705    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253708    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253711    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253713    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253716    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253718    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253721    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253723    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253726    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253728    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253731    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253734    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253737    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253739    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 09:49:22.257323 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253742    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253744    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253747    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253749    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253752    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253755    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253757    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253760    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253764    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253767    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253769    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253772    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253774    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253777    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253780    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253782    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253784    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253787    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253790    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253792    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 09:49:22.257833 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253795    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253797    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253800    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253804    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253807    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253810    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253813    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253816    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253818    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253822    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253825    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253827    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253829    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253832    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253835    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253838    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253843    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253846    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253849    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 09:49:22.258298 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253852    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253855    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253863    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253866    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253868    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253871    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253874    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253876    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253879    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253881    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253884    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253886    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253889    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253892    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253894    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253897    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253899    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253902    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253904    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253909    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 09:49:22.258838 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253911    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.253914    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.253995    2567 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254003    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254016    2567 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254020    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254026    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254030    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254036    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254045    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254048    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254053    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254058    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254063    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254067    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254083    2567 flags.go:64] FLAG: --cgroup-root=""
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254087    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254090    2567 flags.go:64] FLAG: --client-ca-file=""
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254093    2567 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254096    2567 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254102    2567 flags.go:64] FLAG: --cluster-domain=""
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254105    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254108    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254111    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 09:49:22.259574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254115    2567 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254118    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254121    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254125    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254128    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254132    2567 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254134    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254138    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254142    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254144    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254149    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254151    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254154    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254157    2567 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254160    2567 flags.go:64] FLAG: --enable-server="true"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254163    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254170    2567 flags.go:64] FLAG: --event-burst="100"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254175    2567 flags.go:64] FLAG: --event-qps="50"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254178    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254181    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254184    2567 flags.go:64] FLAG: --eviction-hard=""
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254188    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254191    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254194    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254203    2567 flags.go:64] FLAG: --eviction-soft=""
Jul 30 09:49:22.260247 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254206    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254209    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254212    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254215    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254218    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254221    2567 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254224    2567 flags.go:64] FLAG: --feature-gates=""
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254227    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254230    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254233    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254237    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254240    2567 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254243    2567 flags.go:64] FLAG: --help="false"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254245    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254250    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254253    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254256    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254259    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254262    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254265    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254268    2567 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254271    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254274    2567 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 09:49:22.260884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254277    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254282    2567 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254285    2567 flags.go:64] FLAG: --kube-reserved=""
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254288    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254291    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254294    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254297    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254300    2567 flags.go:64] FLAG: --lock-file=""
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254303    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254311    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254314    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254320    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254323    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254325    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254328    2567 flags.go:64] FLAG: --logging-format="text"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254331    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254334    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254337    2567 flags.go:64] FLAG: --manifest-url=""
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254340    2567 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254344    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254347    2567 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254363    2567 flags.go:64] FLAG: --max-pods="110"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254367    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254371    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254374    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 09:49:22.261500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254377    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254380    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254383    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254386    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254394    2567 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254397    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254400    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254403    2567 flags.go:64] FLAG: --pod-cidr=""
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254408    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254414    2567 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254417    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254420    2567 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254423    2567 flags.go:64] FLAG: --port="10250"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254426    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254429    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-021cdd46b62a69458"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254432    2567 flags.go:64] FLAG: --qos-reserved=""
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254436    2567 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254445    2567 flags.go:64] FLAG: --register-node="true"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254448    2567 flags.go:64] FLAG: --register-with-taints=""
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254451    2567 flags.go:64] FLAG: --registry-burst="10"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254454    2567 flags.go:64] FLAG: --registry-qps="5"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254457    2567 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254460    2567 flags.go:64] FLAG: --reserved-memory=""
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254464    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254467    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 09:49:22.262112 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254470    2567 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254472    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254475    2567 flags.go:64] FLAG: --runonce="false"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254478    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254481    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254484    2567 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254488    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254491    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254494    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254497    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254501    2567 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254503    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254506    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254509    2567 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254512    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254517    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254520    2567 flags.go:64] FLAG: --system-cgroups=""
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254523    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254528    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254531    2567 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254534    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254540    2567 flags.go:64] FLAG: --tls-min-version=""
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254543    2567 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254546    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254549    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 09:49:22.262757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254557    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254561    2567 flags.go:64] FLAG: --v="2"
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254565    2567 flags.go:64] FLAG: --version="false"
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254569    2567 flags.go:64] FLAG: --vmodule=""
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254573    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.254576    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254713    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254717    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254720    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254723    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254726    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254729    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254732    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254738    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254747    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254751    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254753    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254757    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254760    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254763    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 09:49:22.263423 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254766    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254768    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254772    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254775    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254778    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254780    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254783    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254785    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254788    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254790    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254793    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254795    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254798    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254801    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254804    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254807    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254809    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254811    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254814    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254817    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 09:49:22.263960 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254819    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254821    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254824    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254826    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254829    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254833    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254835    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254838    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254841    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254843    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254846    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254848    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254851    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254853    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254857    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254860    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254863    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254866    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254869    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254871    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 09:49:22.264474 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254874    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254878    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254881    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254884    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254887    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254890    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254893    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254895    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254898    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254900    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254903    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254905    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254908    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254911    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254913    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254917    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254920    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254924    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254927    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 09:49:22.264997 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254930    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254933    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254935    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254938    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254940    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254943    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254945    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254949    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254951    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254954    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254957    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254959    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254962    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254965    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254967    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254970    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254972    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254975    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254977    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 09:49:22.265563 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254980    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 09:49:22.266041 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254983    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 09:49:22.266041 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254986    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 09:49:22.266041 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254988    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 09:49:22.266041 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254991    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 09:49:22.266041 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254993    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 09:49:22.266041 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.254996    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 09:49:22.266041 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.255005    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 09:49:22.266041 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.261671    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 09:49:22.266041 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.261689    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 09:49:22.266041 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.261710    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 09:49:22.266041 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.261717    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 09:49:22.266041 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261777    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 09:49:22.266041 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261783    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 09:49:22.266041 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261786    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261789    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261791    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261794    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261797    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261799    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261802    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261805    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261807    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261810    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261812    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261815    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261819    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261821    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261824    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261826    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261829    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261832    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261834    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261837    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 09:49:22.266422 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261839    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261843    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261848    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261851    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261853    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261857    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261861    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261866    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261869    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261872    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261875    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261878    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261880    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261883    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261886    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261889    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261891    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261894    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 09:49:22.266916 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261897    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261899    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261902    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261904    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261908    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261910    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261912    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261915    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261917    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261920    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261923    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261925    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261928    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261930    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261933    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261935    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261939    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261943    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261946    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 09:49:22.267374 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261949    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261952    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261956    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261959    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261963    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261965    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261968    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261971    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261974    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261977    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261980    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261982    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261985    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261988    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261990    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261993    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261995    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.261997    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262000    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262003    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 09:49:22.267858 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262005    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262007    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262010    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262013    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262015    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262017    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262021    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262023    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262026    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262029    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262032    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262034    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262037    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262041    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262043    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262046    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262050    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262052    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262055    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 09:49:22.268340 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262058    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.262063    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262188    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262194    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262197    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262200    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262204    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262207    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262209    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262212    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262214    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262217    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262220    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262222    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262224    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 09:49:22.268889 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262227    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262230    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262232    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262234    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262238    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262240    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262243    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262245    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262248    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262250    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262254    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262259    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262262    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262265    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262268    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262271    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262274    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262276    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262279    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262282    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 09:49:22.269263 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262284    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262287    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262289    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262291    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262294    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262297    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262299    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262302    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262304    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262307    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262310    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262313    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262316    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262319    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262321    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262324    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262326    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262329    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262333    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 09:49:22.269907 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262336    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262339    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262342    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262345    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262348    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262365    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262370    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262372    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262375    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262379    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262382    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262384    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262387    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262390    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262393    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262395    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262398    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262400    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262403    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 09:49:22.270450 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262405    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262408    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262410    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262413    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262415    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262418    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262421    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262423    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262426    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262428    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262431    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262434    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262436    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262439    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262442    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262444    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262447    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262449    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262452    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262454    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 09:49:22.270909 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262457    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 09:49:22.271769 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262460    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 09:49:22.271769 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262462    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 09:49:22.271769 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262465    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 09:49:22.271769 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262468    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 09:49:22.271769 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262470    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 09:49:22.271769 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262473    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 09:49:22.271769 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:22.262475    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 09:49:22.271769 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.262480    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 09:49:22.271769 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.262611    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 09:49:22.271769 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.264493    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 09:49:22.271769 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.265672    2567 server.go:1019] "Starting client certificate rotation"
Jul 30 09:49:22.271769 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.265777    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 09:49:22.271769 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.265808    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 09:49:22.272171 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.268059    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 09:49:22.273229 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.273200    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 09:49:22.282640 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.282620    2567 log.go:25] "Validated CRI v1 runtime API"
Jul 30 09:49:22.286445 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.286427    2567 log.go:25] "Validated CRI v1 image API"
Jul 30 09:49:22.287229 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.287213    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 09:49:22.289539 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.289520    2567 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9c89fd96-cf08-4971-9380-0bd7577b47f3:/dev/nvme0n1p4 fd4de278-abb4-4318-b572-19e12aa4f279:/dev/nvme0n1p3]
Jul 30 09:49:22.289605 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.289539    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 09:49:22.293746 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.293639    2567 manager.go:216] Machine: {Timestamp:2026-07-30 09:49:22.293147508 +0000 UTC m=+0.314324339 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101906 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2359729685e81c3debdb75e59ad056 SystemUUID:ec235972-9685-e81c-3deb-db75e59ad056 BootID:e2774e2d-2366-41c9-a266-8556950ccfa8 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3d:c4:33:d7:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3d:c4:33:d7:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:72:ed:e9:4d:89 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 09:49:22.293746 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.293743    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 09:49:22.293855 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.293823    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 09:49:22.297123 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.297098    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 09:49:22.297258 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.297125    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-146.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 09:49:22.297768 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.297757    2567 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 09:49:22.297805 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.297770    2567 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 09:49:22.297805 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.297777    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 09:49:22.298299 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.298290    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 09:49:22.299007 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.298996    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 09:49:22.300600 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.300588    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 09:49:22.300720 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.300710    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 09:49:22.300758 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.300740    2567 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 09:49:22.300758 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.300757    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 09:49:22.300832 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.300771    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 09:49:22.300832 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.300780    2567 kubelet.go:398] "Adding apiserver pod source"
Jul 30 09:49:22.300832 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.300791    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 09:49:22.301141 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.301128    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 09:49:22.301174 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.301154    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 09:49:22.301768 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.301746    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 09:49:22.306226 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.306207    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 09:49:22.307884 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.307870    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 09:49:22.307932 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.307900    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 09:49:22.309228 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.309217    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 09:49:22.309281 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.309232    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 09:49:22.309281 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.309239    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 09:49:22.309281 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.309244    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 09:49:22.309281 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.309252    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 09:49:22.309281 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.309258    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 09:49:22.309281 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.309265    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 09:49:22.309281 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.309271    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 09:49:22.309281 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.309277    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 09:49:22.309281 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.309282    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 09:49:22.309545 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.309291    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 09:49:22.309545 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.309296    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 09:49:22.309989 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.309979    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 09:49:22.309989 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.309987    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 09:49:22.312552 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.312528    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-146.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 09:49:22.313125 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.313102    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 09:49:22.313216 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.313101    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-146.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 09:49:22.313337 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.313326    2567 server.go:1268] "Started kubelet"
Jul 30 09:49:22.313469 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.313417    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 09:49:22.313530 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.313481    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 09:49:22.313576 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.313541    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 09:49:22.314271 ip-10-0-128-146 systemd[1]: Started Kubernetes Kubelet.
Jul 30 09:49:22.315323 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.315305    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 09:49:22.315419 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.315309    2567 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 09:49:22.318331 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.318313    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 09:49:22.321436 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.320537    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-146.ec2.internal.18c7085a8ee767fa  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-146.ec2.internal,UID:ip-10-0-128-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-146.ec2.internal,},FirstTimestamp:2026-07-30 09:49:22.313299962 +0000 UTC m=+0.334476793,LastTimestamp:2026-07-30 09:49:22.313299962 +0000 UTC m=+0.334476793,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-146.ec2.internal,}"
Jul 30 09:49:22.322334 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.322316    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 09:49:22.322434 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.322336    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 09:49:22.322434 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.322426    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 30 09:49:22.322820 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.322802    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 09:49:22.322820 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.322821    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 09:49:22.322940 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.322803    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 09:49:22.322979 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.322946    2567 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 09:49:22.322979 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.322956    2567 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 09:49:22.323210 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.323195    2567 factory.go:55] Registering systemd factory
Jul 30 09:49:22.323210 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.323211    2567 factory.go:223] Registration of the systemd container factory successfully
Jul 30 09:49:22.323472 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.323460    2567 factory.go:153] Registering CRI-O factory
Jul 30 09:49:22.323563 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.323475    2567 factory.go:223] Registration of the crio container factory successfully
Jul 30 09:49:22.323563 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.323537    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 09:49:22.323563 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.323560    2567 factory.go:103] Registering Raw factory
Jul 30 09:49:22.323679 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.323575    2567 manager.go:1196] Started watching for new ooms in manager
Jul 30 09:49:22.324123 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.324108    2567 manager.go:349] Starting recovery of all containers
Jul 30 09:49:22.325965 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.325933    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 30 09:49:22.326115 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.326081    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-146.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 30 09:49:22.334926 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.334902    2567 manager.go:354] Recovery completed
Jul 30 09:49:22.339143 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.339032    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 09:49:22.341398 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.341378    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 09:49:22.341469 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.341413    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 09:49:22.341469 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.341423    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 30 09:49:22.341899 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.341885    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 09:49:22.341899 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.341899    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 09:49:22.341966 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.341914    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 09:49:22.341994 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.341897    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 09:49:22.343382 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.343369    2567 policy_none.go:49] "None policy: Start"
Jul 30 09:49:22.343509 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.343385    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 09:49:22.343509 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.343396    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 09:49:22.343509 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.343495    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 09:49:22.343609 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.343523    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 09:49:22.343609 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.343537    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 30 09:49:22.343767 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.343703    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-146.ec2.internal.18c7085a9094278f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-146.ec2.internal,UID:ip-10-0-128-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-146.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-146.ec2.internal,},FirstTimestamp:2026-07-30 09:49:22.341398415 +0000 UTC m=+0.362575249,LastTimestamp:2026-07-30 09:49:22.341398415 +0000 UTC m=+0.362575249,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-146.ec2.internal,}"
Jul 30 09:49:22.344581 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.344565    2567 policy_none.go:47] "Start"
Jul 30 09:49:22.351763 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.351697    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-146.ec2.internal.18c7085a909473b9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-146.ec2.internal,UID:ip-10-0-128-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-146.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-146.ec2.internal,},FirstTimestamp:2026-07-30 09:49:22.341417913 +0000 UTC m=+0.362594747,LastTimestamp:2026-07-30 09:49:22.341417913 +0000 UTC m=+0.362594747,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-146.ec2.internal,}"
Jul 30 09:49:22.354109 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.354053    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-146.ec2.internal.18c7085a90949713  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-146.ec2.internal,UID:ip-10-0-128-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-146.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-146.ec2.internal,},FirstTimestamp:2026-07-30 09:49:22.341426963 +0000 UTC m=+0.362603796,LastTimestamp:2026-07-30 09:49:22.341426963 +0000 UTC m=+0.362603796,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-146.ec2.internal,}"
Jul 30 09:49:22.355272 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.355256    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vk5nh"
Jul 30 09:49:22.362718 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.362653    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-146.ec2.internal.18c7085a9094278f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-146.ec2.internal.18c7085a9094278f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-146.ec2.internal,UID:ip-10-0-128-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-146.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-146.ec2.internal,},FirstTimestamp:2026-07-30 09:49:22.341398415 +0000 UTC m=+0.362575249,LastTimestamp:2026-07-30 09:49:22.343507646 +0000 UTC m=+0.364684478,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-146.ec2.internal,}"
Jul 30 09:49:22.363180 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.363035    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vk5nh"
Jul 30 09:49:22.372556 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.372489    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-146.ec2.internal.18c7085a909473b9\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-146.ec2.internal.18c7085a909473b9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-146.ec2.internal,UID:ip-10-0-128-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-146.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-146.ec2.internal,},FirstTimestamp:2026-07-30 09:49:22.341417913 +0000 UTC m=+0.362594747,LastTimestamp:2026-07-30 09:49:22.343529977 +0000 UTC m=+0.364706809,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-146.ec2.internal,}"
Jul 30 09:49:22.379952 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.379936    2567 manager.go:337] "Starting Device Plugin manager"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.379968    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.379979    2567 server.go:87] "Starting device plugin registration server"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.380234    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.380246    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.380315    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.380406    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.380415    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.380943    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.380991    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.383584    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-146.ec2.internal.18c7085a90949713\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-146.ec2.internal.18c7085a90949713  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-146.ec2.internal,UID:ip-10-0-128-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-146.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-146.ec2.internal,},FirstTimestamp:2026-07-30 09:49:22.341426963 +0000 UTC m=+0.362603796,LastTimestamp:2026-07-30 09:49:22.343543141 +0000 UTC m=+0.364719972,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-146.ec2.internal,}"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.393325    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-146.ec2.internal.18c7085a93056853  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-146.ec2.internal,UID:ip-10-0-128-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-128-146.ec2.internal,},FirstTimestamp:2026-07-30 09:49:22.382374995 +0000 UTC m=+0.403551817,LastTimestamp:2026-07-30 09:49:22.382374995 +0000 UTC m=+0.403551817,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-146.ec2.internal,}"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.480591    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.481660    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.481693    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.481706    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.481733    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.494916 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.491630    2567 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-146.ec2.internal.18c7085a9094278f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-146.ec2.internal.18c7085a9094278f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-146.ec2.internal,UID:ip-10-0-128-146.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-146.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-146.ec2.internal,},FirstTimestamp:2026-07-30 09:49:22.341398415 +0000 UTC m=+0.362575249,LastTimestamp:2026-07-30 09:49:22.481681073 +0000 UTC m=+0.502857911,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-146.ec2.internal,}"
Jul 30 09:49:22.504464 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.504408    2567 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-128-146.ec2.internal\" not found" node="ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.528468 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.528438    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 09:49:22.529700 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.529684    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 09:49:22.529783 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.529721    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 09:49:22.529783 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.529749    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 09:49:22.529877 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.529789    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 09:49:22.532424 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.532407    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 09:49:22.533192 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.533178    2567 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-146.ec2.internal\" not found" node="ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.630055 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.630007    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal"]
Jul 30 09:49:22.630194 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.630100    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 09:49:22.631643 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.631627    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 09:49:22.631752 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.631656    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 09:49:22.631752 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.631668    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 30 09:49:22.634015 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.634002    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 09:49:22.634132 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.634117    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.634171 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.634154    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 09:49:22.634816 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.634799    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 09:49:22.634924 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.634826    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 09:49:22.634924 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.634836    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 30 09:49:22.634924 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.634881    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 09:49:22.634924 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.634905    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 09:49:22.634924 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.634918    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 30 09:49:22.637116 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.637097    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.637217 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.637130    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 09:49:22.637863 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.637850    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 09:49:22.637942 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.637879    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 09:49:22.637942 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.637904    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 30 09:49:22.658119 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.658096    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-146.ec2.internal\" not found" node="ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.662363 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.662338    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-146.ec2.internal\" not found" node="ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.705483 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.705462    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 09:49:22.706433 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.706414    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 09:49:22.706535 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.706452    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 09:49:22.706535 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.706466    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeHasSufficientPID"
Jul 30 09:49:22.706535 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.706498    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.715637 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.715621    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.715710 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.715644    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-146.ec2.internal\": node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 30 09:49:22.725575 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.725556    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f04a7535bef829f67780274b62654545-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal\" (UID: \"f04a7535bef829f67780274b62654545\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.725671 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.725579    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f04a7535bef829f67780274b62654545-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal\" (UID: \"f04a7535bef829f67780274b62654545\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.725671 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.725601    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/963818eb2a7ff168ce7d8b5094579a8c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-146.ec2.internal\" (UID: \"963818eb2a7ff168ce7d8b5094579a8c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.745067 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.745037    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 30 09:49:22.826704 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.826469    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f04a7535bef829f67780274b62654545-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal\" (UID: \"f04a7535bef829f67780274b62654545\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.826704 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.826524    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f04a7535bef829f67780274b62654545-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal\" (UID: \"f04a7535bef829f67780274b62654545\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.826704 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.826201    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f04a7535bef829f67780274b62654545-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal\" (UID: \"f04a7535bef829f67780274b62654545\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.826704 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.826577    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f04a7535bef829f67780274b62654545-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal\" (UID: \"f04a7535bef829f67780274b62654545\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.826704 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.826642    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/963818eb2a7ff168ce7d8b5094579a8c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-146.ec2.internal\" (UID: \"963818eb2a7ff168ce7d8b5094579a8c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.827041 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.826706    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/963818eb2a7ff168ce7d8b5094579a8c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-146.ec2.internal\" (UID: \"963818eb2a7ff168ce7d8b5094579a8c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.845196 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.845160    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 30 09:49:22.945747 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:22.945704    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 30 09:49:22.960892 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.960864    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:22.965842 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:22.965566    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:23.046803 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:23.046766    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-146.ec2.internal\" not found"
Jul 30 09:49:23.133076 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:23.132998    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 09:49:24.750068 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.750040    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:24.765445 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.765425    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 09:49:24.767382 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.767367    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal"
Jul 30 09:49:24.775344 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.775317    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 09:49:24.792109 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.792080    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 09:49:24.792280 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.792260    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 09:49:24.792361 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.792284    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 09:49:24.828407 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.828380    2567 apiserver.go:52] "Watching apiserver"
Jul 30 09:49:24.841781 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.841753    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 09:49:24.842166 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.842146    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-x52kj","openshift-ovn-kubernetes/ovnkube-node-wf4hv","kube-system/global-pull-secret-syncer-dfptt","kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2","openshift-image-registry/node-ca-lt2xc","openshift-multus/network-metrics-daemon-bf2q6","openshift-network-diagnostics/network-check-target-gxth2","kube-system/konnectivity-agent-gl5sh","openshift-cluster-node-tuning-operator/tuned-lttgm","openshift-dns/node-resolver-gqhzm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal","openshift-multus/multus-additional-cni-plugins-98kcx","openshift-multus/multus-k2btn"]
Jul 30 09:49:24.844673 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.844658    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x52kj"
Jul 30 09:49:24.846753 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.846737    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 09:49:24.846896 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.846878    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 09:49:24.846950 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.846878    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 09:49:24.847055 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.847041    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.847116 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.847062    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-86wz5\""
Jul 30 09:49:24.849134 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.849115    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 09:49:24.849238 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.849221    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:24.849318 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.849302    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 09:49:24.849398 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.849320    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nqnfm\""
Jul 30 09:49:24.849398 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:24.849301    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:24.849494 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.849446    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 09:49:24.849494 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.849461    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 09:49:24.849494 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.849468    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 09:49:24.849624 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.849611    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 09:49:24.849685 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.849669    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 09:49:24.851482 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.851464    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.853521 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.853504    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2955s\""
Jul 30 09:49:24.853642 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.853529    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 09:49:24.853642 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.853539    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 09:49:24.853800 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.853786    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lt2xc"
Jul 30 09:49:24.853834 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.853807    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 09:49:24.855572 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855556    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-cni-netd\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.855634 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855578    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-sys-fs\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.855634 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855596    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c76b8d5-3286-476c-9538-1470c5932ed1-host-slash\") pod \"iptables-alerter-x52kj\" (UID: \"3c76b8d5-3286-476c-9538-1470c5932ed1\") " pod="openshift-network-operator/iptables-alerter-x52kj"
Jul 30 09:49:24.855634 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855625    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-slash\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.855728 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855659    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-node-log\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.855728 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855677    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-log-socket\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.855728 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855692    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.855728 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855710    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-socket-dir\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.855894 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855726    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-run-ovn-kubernetes\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.855894 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855744    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f8st\" (UniqueName: \"kubernetes.io/projected/bafeef50-65d3-4fe3-8c83-c70a40f46d73-kube-api-access-8f8st\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.855894 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855783    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:24.855894 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855817    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-etc-selinux\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.855894 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855863    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-kubelet\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.855894 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855888    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-systemd-units\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.856186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855914    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-run-openvswitch\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.856186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-cni-bin\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.856186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855961    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bafeef50-65d3-4fe3-8c83-c70a40f46d73-ovnkube-script-lib\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.856186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.855986    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/807d7a19-7b79-4963-a013-53d46d57b8a2-kubelet-config\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:24.856186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856012    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3c76b8d5-3286-476c-9538-1470c5932ed1-iptables-alerter-script\") pod \"iptables-alerter-x52kj\" (UID: \"3c76b8d5-3286-476c-9538-1470c5932ed1\") " pod="openshift-network-operator/iptables-alerter-x52kj"
Jul 30 09:49:24.856186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856046    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-run-netns\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.856186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856074    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-var-lib-openvswitch\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.856186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856112    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bafeef50-65d3-4fe3-8c83-c70a40f46d73-ovnkube-config\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.856186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856135    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-run-systemd\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.856186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856158    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-registration-dir\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.856186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856186    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-etc-openvswitch\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.856628 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856207    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bafeef50-65d3-4fe3-8c83-c70a40f46d73-env-overrides\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.856628 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856241    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bafeef50-65d3-4fe3-8c83-c70a40f46d73-ovn-node-metrics-cert\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.856628 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856188    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:24.856628 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856262    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.856628 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856314    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-device-dir\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.856628 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:24.856321    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:24.856628 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856372    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf5tr\" (UniqueName: \"kubernetes.io/projected/6cac85b1-9936-480c-afe4-3dd54fe0bda9-kube-api-access-zf5tr\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.856628 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856402    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/807d7a19-7b79-4963-a013-53d46d57b8a2-dbus\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:24.856628 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856462    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mbb9\" (UniqueName: \"kubernetes.io/projected/3c76b8d5-3286-476c-9538-1470c5932ed1-kube-api-access-5mbb9\") pod \"iptables-alerter-x52kj\" (UID: \"3c76b8d5-3286-476c-9538-1470c5932ed1\") " pod="openshift-network-operator/iptables-alerter-x52kj"
Jul 30 09:49:24.856628 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856485    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 09:49:24.856628 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856492    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-run-ovn\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.857084 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.856536    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 09:49:24.857130 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.857104    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 09:49:24.857213 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.857193    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hlmzz\""
Jul 30 09:49:24.859683 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.859663    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 09:49:24.860244 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.860230    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:24.860295 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:24.860280    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:24.862703 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.862683    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gl5sh"
Jul 30 09:49:24.864967 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.864949    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.865048 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.865025    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 09:49:24.865048 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.865037    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mhgvj\""
Jul 30 09:49:24.865139 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.865053    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 09:49:24.867103 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.867089    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gqhzm"
Jul 30 09:49:24.868158 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.868145    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 09:49:24.868273 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.868254    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 09:49:24.868324 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.868293    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hbpdk\""
Jul 30 09:49:24.869095 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.869083    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 09:49:24.869198 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.869173    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zdjfv\""
Jul 30 09:49:24.869260 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.869249    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 09:49:24.869434 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.869420    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:24.871579 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.871564    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.871687 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.871673    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 09:49:24.871749 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.871675    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 09:49:24.871800 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.871763    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 09:49:24.871922 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.871908    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pkblx\""
Jul 30 09:49:24.872002 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.871989    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 09:49:24.872089 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.872075    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 09:49:24.873559 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.873543    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kr9w5\""
Jul 30 09:49:24.873931 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.873919    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 09:49:24.887230 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.887212    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dcmv2"
Jul 30 09:49:24.895085 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.895067    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dcmv2"
Jul 30 09:49:24.950874 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.950860    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 09:49:24.957300 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957282    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf5tr\" (UniqueName: \"kubernetes.io/projected/6cac85b1-9936-480c-afe4-3dd54fe0bda9-kube-api-access-zf5tr\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.957379 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957311    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-multus-conf-dir\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.957379 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957328    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f556c8a9-3801-4f40-b5d7-1ffa2701eed1-serviceca\") pod \"node-ca-lt2xc\" (UID: \"f556c8a9-3801-4f40-b5d7-1ffa2701eed1\") " pod="openshift-image-registry/node-ca-lt2xc"
Jul 30 09:49:24.957459 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957381    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-run-ovn\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.957459 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957417    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1bbbef53-12ef-475f-934f-5331f026baa6-agent-certs\") pod \"konnectivity-agent-gl5sh\" (UID: \"1bbbef53-12ef-475f-934f-5331f026baa6\") " pod="kube-system/konnectivity-agent-gl5sh"
Jul 30 09:49:24.957528 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957456    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-kubernetes\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.957528 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957477    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-sys\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.957528 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957474    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-run-ovn\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.957528 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957521    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-tuned\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.957684 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957564    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c76b8d5-3286-476c-9538-1470c5932ed1-host-slash\") pod \"iptables-alerter-x52kj\" (UID: \"3c76b8d5-3286-476c-9538-1470c5932ed1\") " pod="openshift-network-operator/iptables-alerter-x52kj"
Jul 30 09:49:24.957684 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957609    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs\") pod \"network-metrics-daemon-bf2q6\" (UID: \"02c86fe5-e1d2-486d-adbc-1ed595aa0f72\") " pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:24.957684 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957644    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c76b8d5-3286-476c-9538-1470c5932ed1-host-slash\") pod \"iptables-alerter-x52kj\" (UID: \"3c76b8d5-3286-476c-9538-1470c5932ed1\") " pod="openshift-network-operator/iptables-alerter-x52kj"
Jul 30 09:49:24.957844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957719    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1bbbef53-12ef-475f-934f-5331f026baa6-konnectivity-ca\") pod \"konnectivity-agent-gl5sh\" (UID: \"1bbbef53-12ef-475f-934f-5331f026baa6\") " pod="kube-system/konnectivity-agent-gl5sh"
Jul 30 09:49:24.957844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957745    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2cfa238d-618e-4dc2-9128-77df904614c9-hosts-file\") pod \"node-resolver-gqhzm\" (UID: \"2cfa238d-618e-4dc2-9128-77df904614c9\") " pod="openshift-dns/node-resolver-gqhzm"
Jul 30 09:49:24.957844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-cnibin\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.957844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957805    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-sysconfig\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.958026 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957861    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-host\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.958026 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957900    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-slash\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.958026 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957933    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.958026 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957962    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-socket-dir\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.958026 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957965    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-slash\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.958026 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.957998    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.958497 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958004    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt6fw\" (UniqueName: \"kubernetes.io/projected/2cfa238d-618e-4dc2-9128-77df904614c9-kube-api-access-nt6fw\") pod \"node-resolver-gqhzm\" (UID: \"2cfa238d-618e-4dc2-9128-77df904614c9\") " pod="openshift-dns/node-resolver-gqhzm"
Jul 30 09:49:24.958548 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958525    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-os-release\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.958581 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958552    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-multus-socket-dir-parent\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.958618 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958577    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e69de26a-c120-4b24-a283-04e2db290ed4-system-cni-dir\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:24.958618 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958606    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f8st\" (UniqueName: \"kubernetes.io/projected/bafeef50-65d3-4fe3-8c83-c70a40f46d73-kube-api-access-8f8st\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.958762 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958630    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-multus-cni-dir\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.958762 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958651    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-modprobe-d\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.958762 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958673    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-sysctl-conf\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.958762 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958708    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-cni-bin\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.958762 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958755    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-cni-bin\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.958967 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958777    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bafeef50-65d3-4fe3-8c83-c70a40f46d73-ovnkube-script-lib\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.958967 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958799    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/807d7a19-7b79-4963-a013-53d46d57b8a2-kubelet-config\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:24.958967 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958797    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-socket-dir\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.958967 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958841    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh4fg\" (UniqueName: \"kubernetes.io/projected/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-kube-api-access-bh4fg\") pod \"network-metrics-daemon-bf2q6\" (UID: \"02c86fe5-e1d2-486d-adbc-1ed595aa0f72\") " pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:24.958967 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958856    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/807d7a19-7b79-4963-a013-53d46d57b8a2-kubelet-config\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:24.958967 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958866    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-hostroot\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.958967 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958909    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e69de26a-c120-4b24-a283-04e2db290ed4-cnibin\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:24.958967 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958932    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3c76b8d5-3286-476c-9538-1470c5932ed1-iptables-alerter-script\") pod \"iptables-alerter-x52kj\" (UID: \"3c76b8d5-3286-476c-9538-1470c5932ed1\") " pod="openshift-network-operator/iptables-alerter-x52kj"
Jul 30 09:49:24.959253 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.958995    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-var-lib-openvswitch\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.959253 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959027    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bafeef50-65d3-4fe3-8c83-c70a40f46d73-ovnkube-config\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.959253 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959052    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-run-multus-certs\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.959253 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959069    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-var-lib-openvswitch\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.959253 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqp5s\" (UniqueName: \"kubernetes.io/projected/64a003d1-48b8-4242-a5bc-526827302072-kube-api-access-gqp5s\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.959253 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959100    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z2cn\" (UniqueName: \"kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn\") pod \"network-check-target-gxth2\" (UID: \"8ade6f03-fb02-44bd-864e-7bdfc5df23b7\") " pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:24.959253 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959127    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-run\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.959253 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959166    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64a003d1-48b8-4242-a5bc-526827302072-cni-binary-copy\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.959580 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959242    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-systemd\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.959580 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959292    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bafeef50-65d3-4fe3-8c83-c70a40f46d73-env-overrides\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.959580 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959357    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bafeef50-65d3-4fe3-8c83-c70a40f46d73-ovn-node-metrics-cert\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.959580 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959388    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f556c8a9-3801-4f40-b5d7-1ffa2701eed1-host\") pod \"node-ca-lt2xc\" (UID: \"f556c8a9-3801-4f40-b5d7-1ffa2701eed1\") " pod="openshift-image-registry/node-ca-lt2xc"
Jul 30 09:49:24.959580 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959405    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/731802bd-8c7b-4b08-b979-9f3eb96972fe-tmp\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.959580 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959428    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn24l\" (UniqueName: \"kubernetes.io/projected/731802bd-8c7b-4b08-b979-9f3eb96972fe-kube-api-access-hn24l\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.959580 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959450    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/807d7a19-7b79-4963-a013-53d46d57b8a2-dbus\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:24.959580 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959494    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mbb9\" (UniqueName: \"kubernetes.io/projected/3c76b8d5-3286-476c-9538-1470c5932ed1-kube-api-access-5mbb9\") pod \"iptables-alerter-x52kj\" (UID: \"3c76b8d5-3286-476c-9538-1470c5932ed1\") " pod="openshift-network-operator/iptables-alerter-x52kj"
Jul 30 09:49:24.959580 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959528    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-cni-netd\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.959580 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959543    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-sys-fs\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.959580 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959570    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-run-k8s-cni-cncf-io\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959605    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e69de26a-c120-4b24-a283-04e2db290ed4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959613    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3c76b8d5-3286-476c-9538-1470c5932ed1-iptables-alerter-script\") pod \"iptables-alerter-x52kj\" (UID: \"3c76b8d5-3286-476c-9538-1470c5932ed1\") " pod="openshift-network-operator/iptables-alerter-x52kj"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959631    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-system-cni-dir\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959634    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/807d7a19-7b79-4963-a013-53d46d57b8a2-dbus\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959647    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-run-netns\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959682    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/64a003d1-48b8-4242-a5bc-526827302072-multus-daemon-config\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959725    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-node-log\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959759    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-log-socket\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959789    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cfa238d-618e-4dc2-9128-77df904614c9-tmp-dir\") pod \"node-resolver-gqhzm\" (UID: \"2cfa238d-618e-4dc2-9128-77df904614c9\") " pod="openshift-dns/node-resolver-gqhzm"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959804    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-node-log\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959810    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-cni-netd\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959812    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-var-lib-cni-bin\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959852    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-log-socket\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959861    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm64h\" (UniqueName: \"kubernetes.io/projected/f556c8a9-3801-4f40-b5d7-1ffa2701eed1-kube-api-access-xm64h\") pod \"node-ca-lt2xc\" (UID: \"f556c8a9-3801-4f40-b5d7-1ffa2701eed1\") " pod="openshift-image-registry/node-ca-lt2xc"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959889    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-run-ovn-kubernetes\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959959    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-sys-fs\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.960070 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.959967    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-run-ovn-kubernetes\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960042    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960067    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-etc-selinux\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960143    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-var-lib-kubelet\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960169    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-var-lib-kubelet\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960179    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960194    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e69de26a-c120-4b24-a283-04e2db290ed4-os-release\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960276    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e69de26a-c120-4b24-a283-04e2db290ed4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960326    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-kubelet\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:24.960368    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960392    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-systemd-units\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960399    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bafeef50-65d3-4fe3-8c83-c70a40f46d73-env-overrides\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960416    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-run-openvswitch\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:24.960474    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret podName:807d7a19-7b79-4963-a013-53d46d57b8a2 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:25.460440567 +0000 UTC m=+1.954636072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret") pod "global-pull-secret-syncer-dfptt" (UID: "807d7a19-7b79-4963-a013-53d46d57b8a2") : object "kube-system"/"original-pull-secret" not registered
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960465    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-run-openvswitch\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960494    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-etc-kubernetes\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960512    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bafeef50-65d3-4fe3-8c83-c70a40f46d73-ovnkube-script-lib\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.960790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960516    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-lib-modules\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.961527 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960842    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-etc-selinux\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.961527 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960559    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-run-netns\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.961527 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961038    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e69de26a-c120-4b24-a283-04e2db290ed4-cni-binary-copy\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:24.961527 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961083    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg4nl\" (UniqueName: \"kubernetes.io/projected/e69de26a-c120-4b24-a283-04e2db290ed4-kube-api-access-jg4nl\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:24.961527 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961124    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-run-systemd\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.961527 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961153    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-registration-dir\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.961527 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961182    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-var-lib-cni-multus\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:24.961527 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961213    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-sysctl-d\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:24.961527 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.960621    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-kubelet\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.961527 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961241    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e69de26a-c120-4b24-a283-04e2db290ed4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:24.961527 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961272    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-etc-openvswitch\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.961527 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961319    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-systemd-units\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.961527 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961387    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bafeef50-65d3-4fe3-8c83-c70a40f46d73-ovnkube-config\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.962083 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961583    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.962083 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961622    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-device-dir\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.962083 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961828    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-registration-dir\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.962083 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961898    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-etc-openvswitch\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.962083 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.961951    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-run-systemd\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.962083 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.962005    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bafeef50-65d3-4fe3-8c83-c70a40f46d73-host-run-netns\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.962083 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.962068    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.962413 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.962089    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6cac85b1-9936-480c-afe4-3dd54fe0bda9-device-dir\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.964519 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.964021    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bafeef50-65d3-4fe3-8c83-c70a40f46d73-ovn-node-metrics-cert\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.967910 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.967887    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf5tr\" (UniqueName: \"kubernetes.io/projected/6cac85b1-9936-480c-afe4-3dd54fe0bda9-kube-api-access-zf5tr\") pod \"aws-ebs-csi-driver-node-kd9q2\" (UID: \"6cac85b1-9936-480c-afe4-3dd54fe0bda9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:24.968263 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.968240    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f8st\" (UniqueName: \"kubernetes.io/projected/bafeef50-65d3-4fe3-8c83-c70a40f46d73-kube-api-access-8f8st\") pod \"ovnkube-node-wf4hv\" (UID: \"bafeef50-65d3-4fe3-8c83-c70a40f46d73\") " pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:24.968400 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:24.968384    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mbb9\" (UniqueName: \"kubernetes.io/projected/3c76b8d5-3286-476c-9538-1470c5932ed1-kube-api-access-5mbb9\") pod \"iptables-alerter-x52kj\" (UID: \"3c76b8d5-3286-476c-9538-1470c5932ed1\") " pod="openshift-network-operator/iptables-alerter-x52kj"
Jul 30 09:49:25.026084 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.025975    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 09:44:23 +0000 UTC" deadline="2028-01-29 23:21:35.05668338 +0000 UTC"
Jul 30 09:49:25.026084 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.026077    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13165h32m10.030609406s"
Jul 30 09:49:25.062152 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062121    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-etc-kubernetes\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.062152 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062153    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-lib-modules\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.062395 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062174    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e69de26a-c120-4b24-a283-04e2db290ed4-cni-binary-copy\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.062395 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062200    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg4nl\" (UniqueName: \"kubernetes.io/projected/e69de26a-c120-4b24-a283-04e2db290ed4-kube-api-access-jg4nl\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.062395 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062227    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-var-lib-cni-multus\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.062395 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062227    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-etc-kubernetes\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.062395 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062249    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-sysctl-d\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.062395 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062288    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e69de26a-c120-4b24-a283-04e2db290ed4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.062395 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062284    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-var-lib-cni-multus\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.062395 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062330    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-lib-modules\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.062395 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062320    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-multus-conf-dir\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.062395 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062370    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-multus-conf-dir\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.062395 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062364    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-sysctl-d\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.062395 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062396    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f556c8a9-3801-4f40-b5d7-1ffa2701eed1-serviceca\") pod \"node-ca-lt2xc\" (UID: \"f556c8a9-3801-4f40-b5d7-1ffa2701eed1\") " pod="openshift-image-registry/node-ca-lt2xc"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062426    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1bbbef53-12ef-475f-934f-5331f026baa6-agent-certs\") pod \"konnectivity-agent-gl5sh\" (UID: \"1bbbef53-12ef-475f-934f-5331f026baa6\") " pod="kube-system/konnectivity-agent-gl5sh"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062449    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-kubernetes\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062471    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-sys\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062491    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-tuned\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062520    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e69de26a-c120-4b24-a283-04e2db290ed4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062534    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs\") pod \"network-metrics-daemon-bf2q6\" (UID: \"02c86fe5-e1d2-486d-adbc-1ed595aa0f72\") " pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062577    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1bbbef53-12ef-475f-934f-5331f026baa6-konnectivity-ca\") pod \"konnectivity-agent-gl5sh\" (UID: \"1bbbef53-12ef-475f-934f-5331f026baa6\") " pod="kube-system/konnectivity-agent-gl5sh"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062584    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-kubernetes\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062604    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2cfa238d-618e-4dc2-9128-77df904614c9-hosts-file\") pod \"node-resolver-gqhzm\" (UID: \"2cfa238d-618e-4dc2-9128-77df904614c9\") " pod="openshift-dns/node-resolver-gqhzm"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062607    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-sys\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062638    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-cnibin\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062662    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-sysconfig\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:25.062669    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062687    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-host\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062715    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt6fw\" (UniqueName: \"kubernetes.io/projected/2cfa238d-618e-4dc2-9128-77df904614c9-kube-api-access-nt6fw\") pod \"node-resolver-gqhzm\" (UID: \"2cfa238d-618e-4dc2-9128-77df904614c9\") " pod="openshift-dns/node-resolver-gqhzm"
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:25.062737    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs podName:02c86fe5-e1d2-486d-adbc-1ed595aa0f72 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:25.562719838 +0000 UTC m=+2.056915333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs") pod "network-metrics-daemon-bf2q6" (UID: "02c86fe5-e1d2-486d-adbc-1ed595aa0f72") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 09:49:25.062947 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062767    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-os-release\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062793    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-multus-socket-dir-parent\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062819    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e69de26a-c120-4b24-a283-04e2db290ed4-system-cni-dir\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062847    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-multus-cni-dir\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062881    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-modprobe-d\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062887    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f556c8a9-3801-4f40-b5d7-1ffa2701eed1-serviceca\") pod \"node-ca-lt2xc\" (UID: \"f556c8a9-3801-4f40-b5d7-1ffa2701eed1\") " pod="openshift-image-registry/node-ca-lt2xc"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062908    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-sysctl-conf\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062942    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh4fg\" (UniqueName: \"kubernetes.io/projected/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-kube-api-access-bh4fg\") pod \"network-metrics-daemon-bf2q6\" (UID: \"02c86fe5-e1d2-486d-adbc-1ed595aa0f72\") " pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062971    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-hostroot\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062989    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2cfa238d-618e-4dc2-9128-77df904614c9-hosts-file\") pod \"node-resolver-gqhzm\" (UID: \"2cfa238d-618e-4dc2-9128-77df904614c9\") " pod="openshift-dns/node-resolver-gqhzm"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063006    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-modprobe-d\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.062881    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e69de26a-c120-4b24-a283-04e2db290ed4-cni-binary-copy\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063029    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e69de26a-c120-4b24-a283-04e2db290ed4-cnibin\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063051    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-multus-socket-dir-parent\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063064    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e69de26a-c120-4b24-a283-04e2db290ed4-system-cni-dir\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063069    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-run-multus-certs\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063107    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-os-release\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063122    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-run-multus-certs\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.063760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063116    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqp5s\" (UniqueName: \"kubernetes.io/projected/64a003d1-48b8-4242-a5bc-526827302072-kube-api-access-gqp5s\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063213    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z2cn\" (UniqueName: \"kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn\") pod \"network-check-target-gxth2\" (UID: \"8ade6f03-fb02-44bd-864e-7bdfc5df23b7\") " pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063240    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-run\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063244    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-sysctl-conf\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063239    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1bbbef53-12ef-475f-934f-5331f026baa6-konnectivity-ca\") pod \"konnectivity-agent-gl5sh\" (UID: \"1bbbef53-12ef-475f-934f-5331f026baa6\") " pod="kube-system/konnectivity-agent-gl5sh"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063283    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64a003d1-48b8-4242-a5bc-526827302072-cni-binary-copy\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063308    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-run\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063326    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-systemd\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063364    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-cnibin\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063368    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-hostroot\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063373    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f556c8a9-3801-4f40-b5d7-1ffa2701eed1-host\") pod \"node-ca-lt2xc\" (UID: \"f556c8a9-3801-4f40-b5d7-1ffa2701eed1\") " pod="openshift-image-registry/node-ca-lt2xc"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063407    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-sysconfig\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063411    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/731802bd-8c7b-4b08-b979-9f3eb96972fe-tmp\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063419    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f556c8a9-3801-4f40-b5d7-1ffa2701eed1-host\") pod \"node-ca-lt2xc\" (UID: \"f556c8a9-3801-4f40-b5d7-1ffa2701eed1\") " pod="openshift-image-registry/node-ca-lt2xc"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063443    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn24l\" (UniqueName: \"kubernetes.io/projected/731802bd-8c7b-4b08-b979-9f3eb96972fe-kube-api-access-hn24l\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063467    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-host\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063476    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-run-k8s-cni-cncf-io\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063495    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-systemd\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.064626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063509    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-run-k8s-cni-cncf-io\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063513    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e69de26a-c120-4b24-a283-04e2db290ed4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063568    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-system-cni-dir\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063583    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-multus-cni-dir\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063714    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-run-netns\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063760    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/64a003d1-48b8-4242-a5bc-526827302072-multus-daemon-config\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.063807    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-run-netns\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064259    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e69de26a-c120-4b24-a283-04e2db290ed4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064287    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64a003d1-48b8-4242-a5bc-526827302072-cni-binary-copy\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064311    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e69de26a-c120-4b24-a283-04e2db290ed4-cnibin\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064313    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/64a003d1-48b8-4242-a5bc-526827302072-multus-daemon-config\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064372    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cfa238d-618e-4dc2-9128-77df904614c9-tmp-dir\") pod \"node-resolver-gqhzm\" (UID: \"2cfa238d-618e-4dc2-9128-77df904614c9\") " pod="openshift-dns/node-resolver-gqhzm"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064432    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-system-cni-dir\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064448    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-var-lib-cni-bin\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064531    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm64h\" (UniqueName: \"kubernetes.io/projected/f556c8a9-3801-4f40-b5d7-1ffa2701eed1-kube-api-access-xm64h\") pod \"node-ca-lt2xc\" (UID: \"f556c8a9-3801-4f40-b5d7-1ffa2701eed1\") " pod="openshift-image-registry/node-ca-lt2xc"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064579    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-var-lib-cni-bin\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064665    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cfa238d-618e-4dc2-9128-77df904614c9-tmp-dir\") pod \"node-resolver-gqhzm\" (UID: \"2cfa238d-618e-4dc2-9128-77df904614c9\") " pod="openshift-dns/node-resolver-gqhzm"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064737    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-var-lib-kubelet\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.065476 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064771    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-var-lib-kubelet\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.065964 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064795    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e69de26a-c120-4b24-a283-04e2db290ed4-os-release\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.065964 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064847    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e69de26a-c120-4b24-a283-04e2db290ed4-os-release\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.065964 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064855    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/731802bd-8c7b-4b08-b979-9f3eb96972fe-var-lib-kubelet\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.065964 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064883    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64a003d1-48b8-4242-a5bc-526827302072-host-var-lib-kubelet\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.065964 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.064891    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e69de26a-c120-4b24-a283-04e2db290ed4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.065964 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.065266    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1bbbef53-12ef-475f-934f-5331f026baa6-agent-certs\") pod \"konnectivity-agent-gl5sh\" (UID: \"1bbbef53-12ef-475f-934f-5331f026baa6\") " pod="kube-system/konnectivity-agent-gl5sh"
Jul 30 09:49:25.065964 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.065265    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/731802bd-8c7b-4b08-b979-9f3eb96972fe-etc-tuned\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.065964 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.065352    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e69de26a-c120-4b24-a283-04e2db290ed4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.065964 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.065561    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/731802bd-8c7b-4b08-b979-9f3eb96972fe-tmp\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.070106 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:25.070088    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 09:49:25.070106 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:25.070107    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 09:49:25.070218 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:25.070116    2567 projected.go:196] Error preparing data for projected volume kube-api-access-6z2cn for pod openshift-network-diagnostics/network-check-target-gxth2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 09:49:25.070218 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:25.070170    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn podName:8ade6f03-fb02-44bd-864e-7bdfc5df23b7 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:25.570152459 +0000 UTC m=+2.064347964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6z2cn" (UniqueName: "kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn") pod "network-check-target-gxth2" (UID: "8ade6f03-fb02-44bd-864e-7bdfc5df23b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 09:49:25.072080 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.072052    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt6fw\" (UniqueName: \"kubernetes.io/projected/2cfa238d-618e-4dc2-9128-77df904614c9-kube-api-access-nt6fw\") pod \"node-resolver-gqhzm\" (UID: \"2cfa238d-618e-4dc2-9128-77df904614c9\") " pod="openshift-dns/node-resolver-gqhzm"
Jul 30 09:49:25.072547 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.072517    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn24l\" (UniqueName: \"kubernetes.io/projected/731802bd-8c7b-4b08-b979-9f3eb96972fe-kube-api-access-hn24l\") pod \"tuned-lttgm\" (UID: \"731802bd-8c7b-4b08-b979-9f3eb96972fe\") " pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.072844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.072825    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm64h\" (UniqueName: \"kubernetes.io/projected/f556c8a9-3801-4f40-b5d7-1ffa2701eed1-kube-api-access-xm64h\") pod \"node-ca-lt2xc\" (UID: \"f556c8a9-3801-4f40-b5d7-1ffa2701eed1\") " pod="openshift-image-registry/node-ca-lt2xc"
Jul 30 09:49:25.072888 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.072870    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg4nl\" (UniqueName: \"kubernetes.io/projected/e69de26a-c120-4b24-a283-04e2db290ed4-kube-api-access-jg4nl\") pod \"multus-additional-cni-plugins-98kcx\" (UID: \"e69de26a-c120-4b24-a283-04e2db290ed4\") " pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.073069 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.073050    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqp5s\" (UniqueName: \"kubernetes.io/projected/64a003d1-48b8-4242-a5bc-526827302072-kube-api-access-gqp5s\") pod \"multus-k2btn\" (UID: \"64a003d1-48b8-4242-a5bc-526827302072\") " pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.073108 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.073079    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh4fg\" (UniqueName: \"kubernetes.io/projected/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-kube-api-access-bh4fg\") pod \"network-metrics-daemon-bf2q6\" (UID: \"02c86fe5-e1d2-486d-adbc-1ed595aa0f72\") " pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:25.119882 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.119853    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 09:49:25.181929 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.181869    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x52kj"
Jul 30 09:49:25.195739 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.195712    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:25.217702 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.217673    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2"
Jul 30 09:49:25.229655 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.229624    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lt2xc"
Jul 30 09:49:25.235866 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.235840    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gl5sh"
Jul 30 09:49:25.260285 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.260256    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lttgm"
Jul 30 09:49:25.290926 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.290902    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gqhzm"
Jul 30 09:49:25.295796 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.295771    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-98kcx"
Jul 30 09:49:25.296889 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.296874    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k2btn"
Jul 30 09:49:25.299264 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:25.299231    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod963818eb2a7ff168ce7d8b5094579a8c.slice/crio-31428844f5356c1ecaf37ab70b33311650e87bc6b2d59810551304c2a5f14956 WatchSource:0}: Error finding container 31428844f5356c1ecaf37ab70b33311650e87bc6b2d59810551304c2a5f14956: Status 404 returned error can't find the container with id 31428844f5356c1ecaf37ab70b33311650e87bc6b2d59810551304c2a5f14956
Jul 30 09:49:25.300196 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:25.300179    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf04a7535bef829f67780274b62654545.slice/crio-5e38524ec9c2f3279de91d254db6765c951ae898e475011bf7a4715fa7f949cc WatchSource:0}: Error finding container 5e38524ec9c2f3279de91d254db6765c951ae898e475011bf7a4715fa7f949cc: Status 404 returned error can't find the container with id 5e38524ec9c2f3279de91d254db6765c951ae898e475011bf7a4715fa7f949cc
Jul 30 09:49:25.308704 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.308681    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 09:49:25.309789 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:25.309764    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cfa238d_618e_4dc2_9128_77df904614c9.slice/crio-0936560fc9bb8b5d007b0915ce7d805975947ef30021229cfec17b559e42d7eb WatchSource:0}: Error finding container 0936560fc9bb8b5d007b0915ce7d805975947ef30021229cfec17b559e42d7eb: Status 404 returned error can't find the container with id 0936560fc9bb8b5d007b0915ce7d805975947ef30021229cfec17b559e42d7eb
Jul 30 09:49:25.310292 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:25.310274    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode69de26a_c120_4b24_a283_04e2db290ed4.slice/crio-ed5516335608c6e2497660faaf0ae4106c63ced9c10dc610167973f066867dec WatchSource:0}: Error finding container ed5516335608c6e2497660faaf0ae4106c63ced9c10dc610167973f066867dec: Status 404 returned error can't find the container with id ed5516335608c6e2497660faaf0ae4106c63ced9c10dc610167973f066867dec
Jul 30 09:49:25.313394 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:25.313376    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a003d1_48b8_4242_a5bc_526827302072.slice/crio-c5359c6a899b22bb57eaeb3f0697a6cacd9354d45df6df2f30debc53738bc0d6 WatchSource:0}: Error finding container c5359c6a899b22bb57eaeb3f0697a6cacd9354d45df6df2f30debc53738bc0d6: Status 404 returned error can't find the container with id c5359c6a899b22bb57eaeb3f0697a6cacd9354d45df6df2f30debc53738bc0d6
Jul 30 09:49:25.445938 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.445856    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 09:49:25.468363 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.468310    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:25.468474 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:25.468457    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 09:49:25.468532 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:25.468521    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret podName:807d7a19-7b79-4963-a013-53d46d57b8a2 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:26.468506152 +0000 UTC m=+2.962701636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret") pod "global-pull-secret-syncer-dfptt" (UID: "807d7a19-7b79-4963-a013-53d46d57b8a2") : object "kube-system"/"original-pull-secret" not registered
Jul 30 09:49:25.521871 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:25.521843    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbafeef50_65d3_4fe3_8c83_c70a40f46d73.slice/crio-87694a47457fd827ab91fbe102b3152cf3338d36991865e402b738cafa8b974d WatchSource:0}: Error finding container 87694a47457fd827ab91fbe102b3152cf3338d36991865e402b738cafa8b974d: Status 404 returned error can't find the container with id 87694a47457fd827ab91fbe102b3152cf3338d36991865e402b738cafa8b974d
Jul 30 09:49:25.533706 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:25.533678    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c76b8d5_3286_476c_9538_1470c5932ed1.slice/crio-0cc646688b4b52b31671bebe371974affd94055e8475af4736defc714b7609e3 WatchSource:0}: Error finding container 0cc646688b4b52b31671bebe371974affd94055e8475af4736defc714b7609e3: Status 404 returned error can't find the container with id 0cc646688b4b52b31671bebe371974affd94055e8475af4736defc714b7609e3
Jul 30 09:49:25.556814 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:25.556791    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bbbef53_12ef_475f_934f_5331f026baa6.slice/crio-5fe6ad576968fe6cb0e6ee320252f84f1161816c837b16ed9ff11500e4c4f565 WatchSource:0}: Error finding container 5fe6ad576968fe6cb0e6ee320252f84f1161816c837b16ed9ff11500e4c4f565: Status 404 returned error can't find the container with id 5fe6ad576968fe6cb0e6ee320252f84f1161816c837b16ed9ff11500e4c4f565
Jul 30 09:49:25.569128 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.569091    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs\") pod \"network-metrics-daemon-bf2q6\" (UID: \"02c86fe5-e1d2-486d-adbc-1ed595aa0f72\") " pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:25.569270 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:25.569239    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 09:49:25.569392 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:25.569310    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs podName:02c86fe5-e1d2-486d-adbc-1ed595aa0f72 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:26.569290377 +0000 UTC m=+3.063485878 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs") pod "network-metrics-daemon-bf2q6" (UID: "02c86fe5-e1d2-486d-adbc-1ed595aa0f72") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 09:49:25.616994 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:25.616962    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod731802bd_8c7b_4b08_b979_9f3eb96972fe.slice/crio-d812cf58f81a4c5c5fafc81da293725cd7e080102c57bca8772cad128fee1110 WatchSource:0}: Error finding container d812cf58f81a4c5c5fafc81da293725cd7e080102c57bca8772cad128fee1110: Status 404 returned error can't find the container with id d812cf58f81a4c5c5fafc81da293725cd7e080102c57bca8772cad128fee1110
Jul 30 09:49:25.617583 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:25.617556    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf556c8a9_3801_4f40_b5d7_1ffa2701eed1.slice/crio-88d3cdfe1aa28ca34cd79df4b07671f19c16bff15f50931a24798f658c5d3a48 WatchSource:0}: Error finding container 88d3cdfe1aa28ca34cd79df4b07671f19c16bff15f50931a24798f658c5d3a48: Status 404 returned error can't find the container with id 88d3cdfe1aa28ca34cd79df4b07671f19c16bff15f50931a24798f658c5d3a48
Jul 30 09:49:25.669574 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.669513    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z2cn\" (UniqueName: \"kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn\") pod \"network-check-target-gxth2\" (UID: \"8ade6f03-fb02-44bd-864e-7bdfc5df23b7\") " pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:25.669731 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:25.669674    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 09:49:25.669731 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:25.669695    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 09:49:25.669731 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:25.669705    2567 projected.go:196] Error preparing data for projected volume kube-api-access-6z2cn for pod openshift-network-diagnostics/network-check-target-gxth2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 09:49:25.669854 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:25.669754    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn podName:8ade6f03-fb02-44bd-864e-7bdfc5df23b7 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:26.669740368 +0000 UTC m=+3.163935852 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6z2cn" (UniqueName: "kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn") pod "network-check-target-gxth2" (UID: "8ade6f03-fb02-44bd-864e-7bdfc5df23b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 09:49:25.898194 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.898075    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 09:44:24 +0000 UTC" deadline="2028-03-16 22:10:41.348186942 +0000 UTC"
Jul 30 09:49:25.898194 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:25.898143    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14292h21m15.450047646s"
Jul 30 09:49:26.039823 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.039794    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 09:49:26.059310 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.059281    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:26.059498 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:26.059423    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:26.072161 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.072098    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal" event={"ID":"963818eb2a7ff168ce7d8b5094579a8c","Type":"ContainerStarted","Data":"31428844f5356c1ecaf37ab70b33311650e87bc6b2d59810551304c2a5f14956"}
Jul 30 09:49:26.074970 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.074939    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lttgm" event={"ID":"731802bd-8c7b-4b08-b979-9f3eb96972fe","Type":"ContainerStarted","Data":"d812cf58f81a4c5c5fafc81da293725cd7e080102c57bca8772cad128fee1110"}
Jul 30 09:49:26.083937 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.083906    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gl5sh" event={"ID":"1bbbef53-12ef-475f-934f-5331f026baa6","Type":"ContainerStarted","Data":"5fe6ad576968fe6cb0e6ee320252f84f1161816c837b16ed9ff11500e4c4f565"}
Jul 30 09:49:26.087929 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.087894    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x52kj" event={"ID":"3c76b8d5-3286-476c-9538-1470c5932ed1","Type":"ContainerStarted","Data":"0cc646688b4b52b31671bebe371974affd94055e8475af4736defc714b7609e3"}
Jul 30 09:49:26.090678 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.090650    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv" event={"ID":"bafeef50-65d3-4fe3-8c83-c70a40f46d73","Type":"ContainerStarted","Data":"87694a47457fd827ab91fbe102b3152cf3338d36991865e402b738cafa8b974d"}
Jul 30 09:49:26.092350 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.092316    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gqhzm" event={"ID":"2cfa238d-618e-4dc2-9128-77df904614c9","Type":"ContainerStarted","Data":"0936560fc9bb8b5d007b0915ce7d805975947ef30021229cfec17b559e42d7eb"}
Jul 30 09:49:26.096840 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.096814    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal" event={"ID":"f04a7535bef829f67780274b62654545","Type":"ContainerStarted","Data":"5e38524ec9c2f3279de91d254db6765c951ae898e475011bf7a4715fa7f949cc"}
Jul 30 09:49:26.101919 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.101894    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lt2xc" event={"ID":"f556c8a9-3801-4f40-b5d7-1ffa2701eed1","Type":"ContainerStarted","Data":"88d3cdfe1aa28ca34cd79df4b07671f19c16bff15f50931a24798f658c5d3a48"}
Jul 30 09:49:26.104560 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.104500    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k2btn" event={"ID":"64a003d1-48b8-4242-a5bc-526827302072","Type":"ContainerStarted","Data":"c5359c6a899b22bb57eaeb3f0697a6cacd9354d45df6df2f30debc53738bc0d6"}
Jul 30 09:49:26.107469 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.107444    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98kcx" event={"ID":"e69de26a-c120-4b24-a283-04e2db290ed4","Type":"ContainerStarted","Data":"ed5516335608c6e2497660faaf0ae4106c63ced9c10dc610167973f066867dec"}
Jul 30 09:49:26.480612 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.479977    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:26.480612 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:26.480135    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 09:49:26.480612 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:26.480205    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret podName:807d7a19-7b79-4963-a013-53d46d57b8a2 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:28.480186418 +0000 UTC m=+4.974381916 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret") pod "global-pull-secret-syncer-dfptt" (UID: "807d7a19-7b79-4963-a013-53d46d57b8a2") : object "kube-system"/"original-pull-secret" not registered
Jul 30 09:49:26.580658 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.580619    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs\") pod \"network-metrics-daemon-bf2q6\" (UID: \"02c86fe5-e1d2-486d-adbc-1ed595aa0f72\") " pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:26.580854 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:26.580805    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 09:49:26.580917 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:26.580867    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs podName:02c86fe5-e1d2-486d-adbc-1ed595aa0f72 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:28.580848943 +0000 UTC m=+5.075044440 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs") pod "network-metrics-daemon-bf2q6" (UID: "02c86fe5-e1d2-486d-adbc-1ed595aa0f72") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 09:49:26.635566 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.635534    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 09:49:26.682745 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.682709    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z2cn\" (UniqueName: \"kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn\") pod \"network-check-target-gxth2\" (UID: \"8ade6f03-fb02-44bd-864e-7bdfc5df23b7\") " pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:26.682967 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:26.682918    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 09:49:26.682967 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:26.682943    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 09:49:26.682967 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:26.682957    2567 projected.go:196] Error preparing data for projected volume kube-api-access-6z2cn for pod openshift-network-diagnostics/network-check-target-gxth2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 09:49:26.683133 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:26.683016    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn podName:8ade6f03-fb02-44bd-864e-7bdfc5df23b7 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:28.682998302 +0000 UTC m=+5.177193788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6z2cn" (UniqueName: "kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn") pod "network-check-target-gxth2" (UID: "8ade6f03-fb02-44bd-864e-7bdfc5df23b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 09:49:26.898702 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.898632    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 09:44:24 +0000 UTC" deadline="2028-02-24 19:50:06.918324255 +0000 UTC"
Jul 30 09:49:26.898702 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:26.898672    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13786h0m40.019657392s"
Jul 30 09:49:27.057436 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:27.057391    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:27.057619 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:27.057537    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:27.058021 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:27.058002    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:27.058125 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:27.058104    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:27.126078 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:27.125123    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2" event={"ID":"6cac85b1-9936-480c-afe4-3dd54fe0bda9","Type":"ContainerStarted","Data":"441c0722d6ceb0724e6ec53f1e982dd5296b5123fdd8cb5e419df774e87a7ec2"}
Jul 30 09:49:28.058466 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:28.058436    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:28.058915 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:28.058567    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:28.500971 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:28.500935    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:28.501167 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:28.501133    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 09:49:28.501238 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:28.501203    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret podName:807d7a19-7b79-4963-a013-53d46d57b8a2 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:32.501182987 +0000 UTC m=+8.995378476 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret") pod "global-pull-secret-syncer-dfptt" (UID: "807d7a19-7b79-4963-a013-53d46d57b8a2") : object "kube-system"/"original-pull-secret" not registered
Jul 30 09:49:28.602177 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:28.602142    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs\") pod \"network-metrics-daemon-bf2q6\" (UID: \"02c86fe5-e1d2-486d-adbc-1ed595aa0f72\") " pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:28.602381 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:28.602305    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 09:49:28.602454 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:28.602383    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs podName:02c86fe5-e1d2-486d-adbc-1ed595aa0f72 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:32.602363783 +0000 UTC m=+9.096559279 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs") pod "network-metrics-daemon-bf2q6" (UID: "02c86fe5-e1d2-486d-adbc-1ed595aa0f72") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 09:49:28.702913 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:28.702844    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z2cn\" (UniqueName: \"kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn\") pod \"network-check-target-gxth2\" (UID: \"8ade6f03-fb02-44bd-864e-7bdfc5df23b7\") " pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:28.703094 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:28.703049    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 09:49:28.703094 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:28.703075    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 09:49:28.703094 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:28.703088    2567 projected.go:196] Error preparing data for projected volume kube-api-access-6z2cn for pod openshift-network-diagnostics/network-check-target-gxth2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 09:49:28.703248 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:28.703147    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn podName:8ade6f03-fb02-44bd-864e-7bdfc5df23b7 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:32.703127852 +0000 UTC m=+9.197323338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6z2cn" (UniqueName: "kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn") pod "network-check-target-gxth2" (UID: "8ade6f03-fb02-44bd-864e-7bdfc5df23b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 09:49:29.057809 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:29.057513    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:29.057972 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:29.057513    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:29.057972 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:29.057893    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:29.057972 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:29.057952    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:30.057559 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:30.057531    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:30.058022 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:30.057654    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:31.057715 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:31.057631    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:31.057715 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:31.057670    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:31.058235 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:31.057803    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:31.058235 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:31.058210    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:32.058047 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:32.058006    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:32.058562 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:32.058155    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:32.535238 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:32.535141    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:32.535430 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:32.535279    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 09:49:32.535430 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:32.535374    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret podName:807d7a19-7b79-4963-a013-53d46d57b8a2 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:40.535326654 +0000 UTC m=+17.029522138 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret") pod "global-pull-secret-syncer-dfptt" (UID: "807d7a19-7b79-4963-a013-53d46d57b8a2") : object "kube-system"/"original-pull-secret" not registered
Jul 30 09:49:32.636894 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:32.636284    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs\") pod \"network-metrics-daemon-bf2q6\" (UID: \"02c86fe5-e1d2-486d-adbc-1ed595aa0f72\") " pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:32.636894 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:32.636493    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 09:49:32.636894 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:32.636553    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs podName:02c86fe5-e1d2-486d-adbc-1ed595aa0f72 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:40.636535281 +0000 UTC m=+17.130730779 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs") pod "network-metrics-daemon-bf2q6" (UID: "02c86fe5-e1d2-486d-adbc-1ed595aa0f72") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 09:49:32.737463 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:32.737388    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z2cn\" (UniqueName: \"kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn\") pod \"network-check-target-gxth2\" (UID: \"8ade6f03-fb02-44bd-864e-7bdfc5df23b7\") " pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:32.737644 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:32.737552    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 09:49:32.737644 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:32.737570    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 09:49:32.737644 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:32.737584    2567 projected.go:196] Error preparing data for projected volume kube-api-access-6z2cn for pod openshift-network-diagnostics/network-check-target-gxth2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 09:49:32.737644 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:32.737644    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn podName:8ade6f03-fb02-44bd-864e-7bdfc5df23b7 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:40.737626616 +0000 UTC m=+17.231822105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6z2cn" (UniqueName: "kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn") pod "network-check-target-gxth2" (UID: "8ade6f03-fb02-44bd-864e-7bdfc5df23b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 09:49:33.057965 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:33.057929    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:33.058156 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:33.058080    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:33.058559 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:33.057929    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:33.058559 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:33.058541    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:34.058057 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:34.057948    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:34.058217 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:34.058103    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:35.057372 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:35.057318    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:35.057559 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:35.057326    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:35.057559 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:35.057459    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:35.057559 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:35.057548    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:36.057964 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:36.057882    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:36.058319 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:36.058011    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:37.057183 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:37.057146    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:37.057363 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:37.057286    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:37.057363 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:37.057353    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:37.057485 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:37.057444    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:38.057317 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:38.057279    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:38.057794 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:38.057428    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:39.057364 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:39.057319    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:39.057364 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:39.057369    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:39.057855 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:39.057475    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:39.057855 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:39.057586    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:40.057253 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:40.057221    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:40.057434 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:40.057344    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:40.596988 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:40.596898    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:40.597145 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:40.597033    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 09:49:40.597145 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:40.597094    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret podName:807d7a19-7b79-4963-a013-53d46d57b8a2 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:56.597075083 +0000 UTC m=+33.091270571 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret") pod "global-pull-secret-syncer-dfptt" (UID: "807d7a19-7b79-4963-a013-53d46d57b8a2") : object "kube-system"/"original-pull-secret" not registered
Jul 30 09:49:40.697577 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:40.697533    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs\") pod \"network-metrics-daemon-bf2q6\" (UID: \"02c86fe5-e1d2-486d-adbc-1ed595aa0f72\") " pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:40.697770 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:40.697681    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 09:49:40.697770 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:40.697743    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs podName:02c86fe5-e1d2-486d-adbc-1ed595aa0f72 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:56.697727807 +0000 UTC m=+33.191923296 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs") pod "network-metrics-daemon-bf2q6" (UID: "02c86fe5-e1d2-486d-adbc-1ed595aa0f72") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 09:49:40.798226 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:40.798187    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z2cn\" (UniqueName: \"kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn\") pod \"network-check-target-gxth2\" (UID: \"8ade6f03-fb02-44bd-864e-7bdfc5df23b7\") " pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:40.798450 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:40.798374    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 09:49:40.798450 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:40.798393    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 09:49:40.798450 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:40.798409    2567 projected.go:196] Error preparing data for projected volume kube-api-access-6z2cn for pod openshift-network-diagnostics/network-check-target-gxth2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 09:49:40.798619 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:40.798465    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn podName:8ade6f03-fb02-44bd-864e-7bdfc5df23b7 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:56.798447398 +0000 UTC m=+33.292642890 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6z2cn" (UniqueName: "kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn") pod "network-check-target-gxth2" (UID: "8ade6f03-fb02-44bd-864e-7bdfc5df23b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 09:49:41.057439 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:41.057404    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:41.057888 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:41.057420    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:41.057888 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:41.057539    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:41.057888 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:41.057636    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:42.057797 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:42.057766    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:42.058376 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:42.057886    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:43.057781 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:43.057745    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:43.057968 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:43.057744    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:43.057968 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:43.057866    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:43.057968 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:43.057940    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:44.057709 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:44.057670    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:44.057895 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:44.057777    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:45.060849 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:45.060064    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:45.060849 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:45.060233    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:45.060849 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:45.060793    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:45.061532 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:45.061499    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:45.172777 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:45.172309    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal" event={"ID":"963818eb2a7ff168ce7d8b5094579a8c","Type":"ContainerStarted","Data":"28122d0f0c90a8bc2ad5cd245d44af35767858a34dd46be0974bbb7d7bd493d5"}
Jul 30 09:49:45.187236 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:45.182236    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lttgm" event={"ID":"731802bd-8c7b-4b08-b979-9f3eb96972fe","Type":"ContainerStarted","Data":"e92a355f3eacb00105066698d526cbb8fe89b12f53bb0d21e55b9ad1b8a4cdec"}
Jul 30 09:49:45.187236 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:45.184214    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 09:49:45.187236 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:45.184728    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv" event={"ID":"bafeef50-65d3-4fe3-8c83-c70a40f46d73","Type":"ContainerStarted","Data":"bb43ff1315ff141ff81a124fde6defafda353a6665c088a56224216f5a479a27"}
Jul 30 09:49:45.187236 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:45.184811    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv" event={"ID":"bafeef50-65d3-4fe3-8c83-c70a40f46d73","Type":"ContainerStarted","Data":"a6b1d98b0cff72f3eb2a3b1b53b1a847e8bc1766fcc016e1e5ac25d690b4d77c"}
Jul 30 09:49:45.188212 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:45.188145    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-146.ec2.internal" podStartSLOduration=21.188127355 podStartE2EDuration="21.188127355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:49:45.187518099 +0000 UTC m=+21.681713605" watchObservedRunningTime="2026-07-30 09:49:45.188127355 +0000 UTC m=+21.682322863"
Jul 30 09:49:45.228307 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:45.227976    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lttgm" podStartSLOduration=1.9261323 podStartE2EDuration="21.227951514s" totalImagesPullingTime="19.301819214s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:49:45.204904521 +0000 UTC m=+21.699100027" watchObservedRunningTime="2026-07-30 09:49:45.227951514 +0000 UTC m=+21.722147021"
Jul 30 09:49:45.228500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:45.228448    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k2btn" podStartSLOduration=1.463319035 podStartE2EDuration="21.228433119s" totalImagesPullingTime="19.765114084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:49:45.227015547 +0000 UTC m=+21.721211057" watchObservedRunningTime="2026-07-30 09:49:45.228433119 +0000 UTC m=+21.722628626"
Jul 30 09:49:46.058066 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.057888    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:46.058229 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:46.058139    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:46.190378 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.190286    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k2btn" event={"ID":"64a003d1-48b8-4242-a5bc-526827302072","Type":"ContainerStarted","Data":"dd042d6c51295f7e76709e2e6db006571d281319dc4695adc463f505074df928"}
Jul 30 09:49:46.191952 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.191923    2567 generic.go:356] "Generic (PLEG): container finished" podID="e69de26a-c120-4b24-a283-04e2db290ed4" containerID="afc3be63bcde3e7b0063fe1395f5c98228b17f84e0ed6acef4e63bad82cee771" exitCode=0
Jul 30 09:49:46.192084 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.191988    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98kcx" event={"ID":"e69de26a-c120-4b24-a283-04e2db290ed4","Type":"ContainerDied","Data":"afc3be63bcde3e7b0063fe1395f5c98228b17f84e0ed6acef4e63bad82cee771"}
Jul 30 09:49:46.193715 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.193655    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gl5sh" event={"ID":"1bbbef53-12ef-475f-934f-5331f026baa6","Type":"ContainerStarted","Data":"24d5ddcc620ab6f2beef276888507c65e8b3e21c42c147d3ef9142144c7773d3"}
Jul 30 09:49:46.195556 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.195534    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x52kj" event={"ID":"3c76b8d5-3286-476c-9538-1470c5932ed1","Type":"ContainerStarted","Data":"22450d429f1a40269ab673f5357108f28a074b124ecf0bd93790f3e2f04612a8"}
Jul 30 09:49:46.198519 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.198500    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 09:49:46.198866 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.198845    2567 generic.go:356] "Generic (PLEG): container finished" podID="bafeef50-65d3-4fe3-8c83-c70a40f46d73" containerID="bb43ff1315ff141ff81a124fde6defafda353a6665c088a56224216f5a479a27" exitCode=1
Jul 30 09:49:46.198966 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.198921    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv" event={"ID":"bafeef50-65d3-4fe3-8c83-c70a40f46d73","Type":"ContainerDied","Data":"bb43ff1315ff141ff81a124fde6defafda353a6665c088a56224216f5a479a27"}
Jul 30 09:49:46.199031 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.199011    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv" event={"ID":"bafeef50-65d3-4fe3-8c83-c70a40f46d73","Type":"ContainerStarted","Data":"7d9b1904d7dfd09cca6eb47dfd0babbdc89c47d9c09ecedae9c03a101dd41592"}
Jul 30 09:49:46.199091 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.199066    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv" event={"ID":"bafeef50-65d3-4fe3-8c83-c70a40f46d73","Type":"ContainerStarted","Data":"498f8a3e7ad9c078c8bd0744387bebb3d8327fec88be8da3c61309dba910f7a7"}
Jul 30 09:49:46.199145 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.199116    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv" event={"ID":"bafeef50-65d3-4fe3-8c83-c70a40f46d73","Type":"ContainerStarted","Data":"8bfbe5e4f4afe36be05e3dc58593d969bbfef00af5afe163ae9f10aaef137cd2"}
Jul 30 09:49:46.199199 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.199166    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv" event={"ID":"bafeef50-65d3-4fe3-8c83-c70a40f46d73","Type":"ContainerStarted","Data":"c1ff5d48c5263c55644d5da6863854112ac38ee99f112c1ca755018f608510a7"}
Jul 30 09:49:46.200420 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.200358    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gqhzm" event={"ID":"2cfa238d-618e-4dc2-9128-77df904614c9","Type":"ContainerStarted","Data":"981248a7a157d4fe7077713727c316e1a6c67b410c0e391de1007938f70ee401"}
Jul 30 09:49:46.202218 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.202191    2567 generic.go:356] "Generic (PLEG): container finished" podID="f04a7535bef829f67780274b62654545" containerID="d61dd380bd76ba2b5763664a56fdf2dbad8ad66251f6ff0750dbe3514c141820" exitCode=0
Jul 30 09:49:46.202322 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.202288    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal" event={"ID":"f04a7535bef829f67780274b62654545","Type":"ContainerDied","Data":"d61dd380bd76ba2b5763664a56fdf2dbad8ad66251f6ff0750dbe3514c141820"}
Jul 30 09:49:46.203949 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.203929    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2" event={"ID":"6cac85b1-9936-480c-afe4-3dd54fe0bda9","Type":"ContainerStarted","Data":"3d5f9e267254e45bde59eeb6145361b130cdb05bdcd970bada3cd9dc2ae63614"}
Jul 30 09:49:46.205997 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.205970    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lt2xc" event={"ID":"f556c8a9-3801-4f40-b5d7-1ffa2701eed1","Type":"ContainerStarted","Data":"2a2be7e1a7a8ad56df827dcc357af0746be9664921ea753b72938b002b789cdb"}
Jul 30 09:49:46.240822 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.240783    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gl5sh" podStartSLOduration=7.487113293 podStartE2EDuration="22.240763895s" totalImagesPullingTime="14.753650602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:49:46.226786415 +0000 UTC m=+22.720981922" watchObservedRunningTime="2026-07-30 09:49:46.240763895 +0000 UTC m=+22.734959406"
Jul 30 09:49:46.240978 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.240933    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gqhzm" podStartSLOduration=2.658625507 podStartE2EDuration="22.240925332s" totalImagesPullingTime="19.582299825s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:49:46.240594159 +0000 UTC m=+22.734789665" watchObservedRunningTime="2026-07-30 09:49:46.240925332 +0000 UTC m=+22.735120839"
Jul 30 09:49:46.268153 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.268096    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lt2xc" podStartSLOduration=2.993341708 podStartE2EDuration="22.268074634s" totalImagesPullingTime="19.274732926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:49:46.267425428 +0000 UTC m=+22.761620946" watchObservedRunningTime="2026-07-30 09:49:46.268074634 +0000 UTC m=+22.762270139"
Jul 30 09:49:46.990994 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:46.990966    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 09:49:47.057990 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:47.057929    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:47.058154 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:47.057929    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:47.058154 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:47.058039    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:47.058154 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:47.058100    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:47.211589 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:47.211426    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal" event={"ID":"f04a7535bef829f67780274b62654545","Type":"ContainerStarted","Data":"f4e5a4a21c05527592625d24cef1bf408f7c219a9c6b73c363bc591fde33c8ed"}
Jul 30 09:49:47.213469 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:47.213436    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2" event={"ID":"6cac85b1-9936-480c-afe4-3dd54fe0bda9","Type":"ContainerStarted","Data":"ff62daa646e5e7c5c9486283055c9c0f94f38574b84a1645d89bb62f8633cc51"}
Jul 30 09:49:47.247546 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:47.247500    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-146.ec2.internal" podStartSLOduration=23.247482348 podStartE2EDuration="23.247482348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:49:47.231660524 +0000 UTC m=+23.725856031" watchObservedRunningTime="2026-07-30 09:49:47.247482348 +0000 UTC m=+23.741677853"
Jul 30 09:49:47.247726 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:47.247710    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x52kj" podStartSLOduration=3.8621875169999997 podStartE2EDuration="23.247705451s" totalImagesPullingTime="19.385517934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:49:47.247118198 +0000 UTC m=+23.741313704" watchObservedRunningTime="2026-07-30 09:49:47.247705451 +0000 UTC m=+23.741900957"
Jul 30 09:49:47.921354 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:47.921223    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T09:49:46.990984824Z","UUID":"a079d5af-cddb-46a8-8011-e9abe9f702fe","Handler":null,"Name":"","Endpoint":""}
Jul 30 09:49:47.923031 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:47.923005    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 09:49:47.923216 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:47.923054    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 09:49:48.057226 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:48.057197    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:48.057410 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:48.057328    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:48.219473 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:48.219443    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 09:49:48.219933 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:48.219902    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv" event={"ID":"bafeef50-65d3-4fe3-8c83-c70a40f46d73","Type":"ContainerStarted","Data":"b8e366a88cf16c502fe87a4e9179b67ca91a8b867f97fdc38018aa35d2428e1f"}
Jul 30 09:49:49.057354 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:49.057104    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:49.057547 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:49.057140    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:49.057547 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:49.057466    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:49.057547 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:49.057526    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:49.223853 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:49.223821    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2" event={"ID":"6cac85b1-9936-480c-afe4-3dd54fe0bda9","Type":"ContainerStarted","Data":"fc6254bb7c21961bc7181184a84064779aae84fbecca4b768c7a7923d164a85f"}
Jul 30 09:49:50.057589 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:50.057552    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:50.057759 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:50.057675    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:50.739316 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:50.739281    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gl5sh"
Jul 30 09:49:50.739948 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:50.739859    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gl5sh"
Jul 30 09:49:50.753994 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:50.753951    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kd9q2" podStartSLOduration=6.003913275 podStartE2EDuration="26.75393452s" totalImagesPullingTime="20.750021245s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:49:49.241731021 +0000 UTC m=+25.735926527" watchObservedRunningTime="2026-07-30 09:49:50.75393452 +0000 UTC m=+27.248130023"
Jul 30 09:49:51.057535 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:51.057288    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:51.057661 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:51.057303    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:51.057661 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:51.057602    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:51.057661 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:51.057632    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:51.231200 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:51.231164    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98kcx" event={"ID":"e69de26a-c120-4b24-a283-04e2db290ed4","Type":"ContainerStarted","Data":"be1248a206d4b793f78dd414be28215f1700564825e899c37cc8be950627e982"}
Jul 30 09:49:51.234550 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:51.234526    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 09:49:51.234984 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:51.234945    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv" event={"ID":"bafeef50-65d3-4fe3-8c83-c70a40f46d73","Type":"ContainerStarted","Data":"4d60e1d04f637a2c63216fe20ff1f6bdf345cd610f554d892c83945dd2d48c35"}
Jul 30 09:49:51.235096 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:51.235053    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gl5sh"
Jul 30 09:49:51.235493 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:51.235475    2567 scope.go:117] "RemoveContainer" containerID="bb43ff1315ff141ff81a124fde6defafda353a6665c088a56224216f5a479a27"
Jul 30 09:49:51.235615 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:51.235601    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gl5sh"
Jul 30 09:49:52.057939 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.057870    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:52.058552 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:52.057973    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:52.240919 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.240891    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 09:49:52.241388 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.241353    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv" event={"ID":"bafeef50-65d3-4fe3-8c83-c70a40f46d73","Type":"ContainerStarted","Data":"9d0357ab3a87d97ff208c16931147c893560d3629d3b3dc717e7c05b88fe904c"}
Jul 30 09:49:52.241555 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.241535    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:52.241669 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.241587    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:52.241669 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.241623    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:52.243589 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.243565    2567 generic.go:356] "Generic (PLEG): container finished" podID="e69de26a-c120-4b24-a283-04e2db290ed4" containerID="be1248a206d4b793f78dd414be28215f1700564825e899c37cc8be950627e982" exitCode=0
Jul 30 09:49:52.243682 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.243653    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98kcx" event={"ID":"e69de26a-c120-4b24-a283-04e2db290ed4","Type":"ContainerDied","Data":"be1248a206d4b793f78dd414be28215f1700564825e899c37cc8be950627e982"}
Jul 30 09:49:52.257887 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.257867    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:52.258000 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.257962    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:49:52.268028 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.267984    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv" podStartSLOduration=8.840001425 podStartE2EDuration="28.267967546s" totalImagesPullingTime="19.427966121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:49:52.266542059 +0000 UTC m=+28.760737565" watchObservedRunningTime="2026-07-30 09:49:52.267967546 +0000 UTC m=+28.762163051"
Jul 30 09:49:52.892851 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.892755    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dfptt"]
Jul 30 09:49:52.893051 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.893028    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:52.893329 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:52.893281    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:52.896246 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.896058    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bf2q6"]
Jul 30 09:49:52.896246 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.896199    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:52.896463 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:52.896305    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:52.896831 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.896800    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gxth2"]
Jul 30 09:49:52.896917 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:52.896905    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:52.897009 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:52.896985    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:54.058620 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:54.058588    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:54.059008 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:54.058680    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:54.250129 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:54.250093    2567 generic.go:356] "Generic (PLEG): container finished" podID="e69de26a-c120-4b24-a283-04e2db290ed4" containerID="bb1d85ec953d78956fef838663715936a781e5b9207e94199536d17e363ab11b" exitCode=0
Jul 30 09:49:54.250283 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:54.250158    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98kcx" event={"ID":"e69de26a-c120-4b24-a283-04e2db290ed4","Type":"ContainerDied","Data":"bb1d85ec953d78956fef838663715936a781e5b9207e94199536d17e363ab11b"}
Jul 30 09:49:55.057129 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:55.056936    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:55.057313 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:55.056951    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:55.057313 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:55.057207    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:55.057437 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:55.057306    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:56.057592 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:56.057560    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:56.058067 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:56.057675    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfptt" podUID="807d7a19-7b79-4963-a013-53d46d57b8a2"
Jul 30 09:49:56.256229 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:56.256196    2567 generic.go:356] "Generic (PLEG): container finished" podID="e69de26a-c120-4b24-a283-04e2db290ed4" containerID="5c5c928e6f16a0cbdd83043e27f6007e33bc6c57bf2c1009be21dd630f00fdce" exitCode=0
Jul 30 09:49:56.256409 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:56.256255    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98kcx" event={"ID":"e69de26a-c120-4b24-a283-04e2db290ed4","Type":"ContainerDied","Data":"5c5c928e6f16a0cbdd83043e27f6007e33bc6c57bf2c1009be21dd630f00fdce"}
Jul 30 09:49:56.613229 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:56.613193    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:56.613408 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:56.613323    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 09:49:56.613408 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:56.613393    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret podName:807d7a19-7b79-4963-a013-53d46d57b8a2 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:28.613378973 +0000 UTC m=+65.107574456 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret") pod "global-pull-secret-syncer-dfptt" (UID: "807d7a19-7b79-4963-a013-53d46d57b8a2") : object "kube-system"/"original-pull-secret" not registered
Jul 30 09:49:56.713920 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:56.713886    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs\") pod \"network-metrics-daemon-bf2q6\" (UID: \"02c86fe5-e1d2-486d-adbc-1ed595aa0f72\") " pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:56.714060 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:56.713991    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 09:49:56.714060 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:56.714043    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs podName:02c86fe5-e1d2-486d-adbc-1ed595aa0f72 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:28.714030162 +0000 UTC m=+65.208225647 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs") pod "network-metrics-daemon-bf2q6" (UID: "02c86fe5-e1d2-486d-adbc-1ed595aa0f72") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 09:49:56.814644 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:56.814602    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z2cn\" (UniqueName: \"kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn\") pod \"network-check-target-gxth2\" (UID: \"8ade6f03-fb02-44bd-864e-7bdfc5df23b7\") " pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:56.814811 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:56.814754    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 09:49:56.814811 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:56.814775    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 09:49:56.814811 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:56.814787    2567 projected.go:196] Error preparing data for projected volume kube-api-access-6z2cn for pod openshift-network-diagnostics/network-check-target-gxth2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 09:49:56.814954 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:56.814853    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn podName:8ade6f03-fb02-44bd-864e-7bdfc5df23b7 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:28.814834732 +0000 UTC m=+65.309030220 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6z2cn" (UniqueName: "kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn") pod "network-check-target-gxth2" (UID: "8ade6f03-fb02-44bd-864e-7bdfc5df23b7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 09:49:57.057230 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.057194    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:57.057439 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.057235    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:57.057439 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:57.057348    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bf2q6" podUID="02c86fe5-e1d2-486d-adbc-1ed595aa0f72"
Jul 30 09:49:57.057569 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:57.057455    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gxth2" podUID="8ade6f03-fb02-44bd-864e-7bdfc5df23b7"
Jul 30 09:49:57.801715 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.801627    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-146.ec2.internal" event="NodeReady"
Jul 30 09:49:57.802146 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.801786    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 09:49:57.839508 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.839469    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-598bfb974b-kzqfn"]
Jul 30 09:49:57.843266 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.843235    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:57.845817 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.845789    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 30 09:49:57.846052 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.846031    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6fk8k\""
Jul 30 09:49:57.846178 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.846063    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 30 09:49:57.846178 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.846089    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 30 09:49:57.852787 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.852576    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 30 09:49:57.855968 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.855943    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-598bfb974b-kzqfn"]
Jul 30 09:49:57.859260 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.859239    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-84b5h"]
Jul 30 09:49:57.862896 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.862876    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zhg8c"]
Jul 30 09:49:57.863029 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.863009    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-84b5h"
Jul 30 09:49:57.865989 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.865914    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 09:49:57.865989 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.865967    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cw2kw\""
Jul 30 09:49:57.866533 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.865969    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 09:49:57.866533 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.866503    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zhg8c"
Jul 30 09:49:57.869673 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.869489    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 09:49:57.869673 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.869529    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 09:49:57.869673 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.869608    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 09:49:57.869673 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.869652    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s28zx\""
Jul 30 09:49:57.875964 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.875936    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-84b5h"]
Jul 30 09:49:57.876961 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.876940    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zhg8c"]
Jul 30 09:49:57.922741 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.922703    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f923caa-9667-4c0d-8083-47944d143e9d-installation-pull-secrets\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:57.922931 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.922758    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f923caa-9667-4c0d-8083-47944d143e9d-trusted-ca\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:57.922931 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.922791    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:57.922931 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.922818    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f923caa-9667-4c0d-8083-47944d143e9d-image-registry-private-configuration\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:57.922931 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.922835    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f923caa-9667-4c0d-8083-47944d143e9d-ca-trust-extracted\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:57.922931 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.922849    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccdsq\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-kube-api-access-ccdsq\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:57.922931 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.922880    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f923caa-9667-4c0d-8083-47944d143e9d-registry-certificates\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:57.922931 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:57.922904    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-bound-sa-token\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.024205 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024167    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c1a1aad-170c-4e52-8f7d-e763663b81e4-tmp-dir\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:49:58.024398 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024220    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f923caa-9667-4c0d-8083-47944d143e9d-installation-pull-secrets\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.024398 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024250    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c1a1aad-170c-4e52-8f7d-e763663b81e4-config-volume\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:49:58.024398 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024312    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8lxm\" (UniqueName: \"kubernetes.io/projected/3c1a1aad-170c-4e52-8f7d-e763663b81e4-kube-api-access-w8lxm\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:49:58.024398 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024356    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f923caa-9667-4c0d-8083-47944d143e9d-trusted-ca\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.024625 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024409    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw7hx\" (UniqueName: \"kubernetes.io/projected/c89418e7-06ea-4bb2-8c58-22fbd57034b5-kube-api-access-sw7hx\") pod \"ingress-canary-zhg8c\" (UID: \"c89418e7-06ea-4bb2-8c58-22fbd57034b5\") " pod="openshift-ingress-canary/ingress-canary-zhg8c"
Jul 30 09:49:58.024625 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024456    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.024625 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024489    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f923caa-9667-4c0d-8083-47944d143e9d-image-registry-private-configuration\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.024625 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024521    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f923caa-9667-4c0d-8083-47944d143e9d-ca-trust-extracted\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.024625 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:58.024581    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 09:49:58.024625 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:58.024598    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-598bfb974b-kzqfn: secret "image-registry-tls" not found
Jul 30 09:49:58.024863 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:58.024662    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls podName:2f923caa-9667-4c0d-8083-47944d143e9d nodeName:}" failed. No retries permitted until 2026-07-30 09:49:58.524637973 +0000 UTC m=+35.018833459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls") pod "image-registry-598bfb974b-kzqfn" (UID: "2f923caa-9667-4c0d-8083-47944d143e9d") : secret "image-registry-tls" not found
Jul 30 09:49:58.024863 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024683    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccdsq\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-kube-api-access-ccdsq\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.024863 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024716    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:49:58.024863 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024744    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c89418e7-06ea-4bb2-8c58-22fbd57034b5-cert\") pod \"ingress-canary-zhg8c\" (UID: \"c89418e7-06ea-4bb2-8c58-22fbd57034b5\") " pod="openshift-ingress-canary/ingress-canary-zhg8c"
Jul 30 09:49:58.024863 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024791    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f923caa-9667-4c0d-8083-47944d143e9d-registry-certificates\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.024863 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024819    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-bound-sa-token\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.025157 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.024953    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f923caa-9667-4c0d-8083-47944d143e9d-ca-trust-extracted\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.025354 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.025315    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f923caa-9667-4c0d-8083-47944d143e9d-trusted-ca\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.025551 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.025477    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f923caa-9667-4c0d-8083-47944d143e9d-registry-certificates\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.029186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.029153    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f923caa-9667-4c0d-8083-47944d143e9d-image-registry-private-configuration\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.029186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.029170    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f923caa-9667-4c0d-8083-47944d143e9d-installation-pull-secrets\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.035910 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.035882    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-bound-sa-token\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.036035 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.035954    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccdsq\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-kube-api-access-ccdsq\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.038184 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.038147    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log" limit=1024
Jul 30 09:49:58.058228 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.058144    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:49:58.061310 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.061286    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 09:49:58.126040 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.126007    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw7hx\" (UniqueName: \"kubernetes.io/projected/c89418e7-06ea-4bb2-8c58-22fbd57034b5-kube-api-access-sw7hx\") pod \"ingress-canary-zhg8c\" (UID: \"c89418e7-06ea-4bb2-8c58-22fbd57034b5\") " pod="openshift-ingress-canary/ingress-canary-zhg8c"
Jul 30 09:49:58.126218 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.126065    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:49:58.126218 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.126095    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c89418e7-06ea-4bb2-8c58-22fbd57034b5-cert\") pod \"ingress-canary-zhg8c\" (UID: \"c89418e7-06ea-4bb2-8c58-22fbd57034b5\") " pod="openshift-ingress-canary/ingress-canary-zhg8c"
Jul 30 09:49:58.126218 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.126151    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c1a1aad-170c-4e52-8f7d-e763663b81e4-tmp-dir\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:49:58.126218 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.126169    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c1a1aad-170c-4e52-8f7d-e763663b81e4-config-volume\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:49:58.126218 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.126195    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8lxm\" (UniqueName: \"kubernetes.io/projected/3c1a1aad-170c-4e52-8f7d-e763663b81e4-kube-api-access-w8lxm\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:49:58.126530 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:58.126506    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 09:49:58.126607 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:58.126595    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls podName:3c1a1aad-170c-4e52-8f7d-e763663b81e4 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:58.626573186 +0000 UTC m=+35.120768675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls") pod "dns-default-84b5h" (UID: "3c1a1aad-170c-4e52-8f7d-e763663b81e4") : secret "dns-default-metrics-tls" not found
Jul 30 09:49:58.126818 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.126793    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c1a1aad-170c-4e52-8f7d-e763663b81e4-tmp-dir\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:49:58.127076 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.127050    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c1a1aad-170c-4e52-8f7d-e763663b81e4-config-volume\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:49:58.129284 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.129259    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c89418e7-06ea-4bb2-8c58-22fbd57034b5-cert\") pod \"ingress-canary-zhg8c\" (UID: \"c89418e7-06ea-4bb2-8c58-22fbd57034b5\") " pod="openshift-ingress-canary/ingress-canary-zhg8c"
Jul 30 09:49:58.134942 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.134891    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log" limit=1024
Jul 30 09:49:58.138033 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.138006    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8lxm\" (UniqueName: \"kubernetes.io/projected/3c1a1aad-170c-4e52-8f7d-e763663b81e4-kube-api-access-w8lxm\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:49:58.138167 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.138083    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw7hx\" (UniqueName: \"kubernetes.io/projected/c89418e7-06ea-4bb2-8c58-22fbd57034b5-kube-api-access-sw7hx\") pod \"ingress-canary-zhg8c\" (UID: \"c89418e7-06ea-4bb2-8c58-22fbd57034b5\") " pod="openshift-ingress-canary/ingress-canary-zhg8c"
Jul 30 09:49:58.185349 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.185292    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zhg8c"
Jul 30 09:49:58.341317 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.340998    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zhg8c"]
Jul 30 09:49:58.346052 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:49:58.346016    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc89418e7_06ea_4bb2_8c58_22fbd57034b5.slice/crio-aa752f47e997b0ebdf4fda93a13d3699749e05ee323df7c8fb269af7a32c9cd7 WatchSource:0}: Error finding container aa752f47e997b0ebdf4fda93a13d3699749e05ee323df7c8fb269af7a32c9cd7: Status 404 returned error can't find the container with id aa752f47e997b0ebdf4fda93a13d3699749e05ee323df7c8fb269af7a32c9cd7
Jul 30 09:49:58.529880 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.529843    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:58.530045 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:58.530027    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 09:49:58.530086 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:58.530051    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-598bfb974b-kzqfn: secret "image-registry-tls" not found
Jul 30 09:49:58.530130 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:58.530119    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls podName:2f923caa-9667-4c0d-8083-47944d143e9d nodeName:}" failed. No retries permitted until 2026-07-30 09:49:59.530098257 +0000 UTC m=+36.024293744 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls") pod "image-registry-598bfb974b-kzqfn" (UID: "2f923caa-9667-4c0d-8083-47944d143e9d") : secret "image-registry-tls" not found
Jul 30 09:49:58.630874 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:58.630842    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:49:58.631054 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:58.631003    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 09:49:58.631106 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:58.631078    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls podName:3c1a1aad-170c-4e52-8f7d-e763663b81e4 nodeName:}" failed. No retries permitted until 2026-07-30 09:49:59.631060054 +0000 UTC m=+36.125255541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls") pod "dns-default-84b5h" (UID: "3c1a1aad-170c-4e52-8f7d-e763663b81e4") : secret "dns-default-metrics-tls" not found
Jul 30 09:49:59.057233 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:59.057146    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:49:59.057899 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:59.057146    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:49:59.061085 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:59.061060    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 09:49:59.061226 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:59.061133    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 09:49:59.061410 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:59.061389    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 09:49:59.061410 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:59.061403    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l5pnh\""
Jul 30 09:49:59.061549 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:59.061393    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fxw5q\""
Jul 30 09:49:59.265355 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:59.265303    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zhg8c" event={"ID":"c89418e7-06ea-4bb2-8c58-22fbd57034b5","Type":"ContainerStarted","Data":"aa752f47e997b0ebdf4fda93a13d3699749e05ee323df7c8fb269af7a32c9cd7"}
Jul 30 09:49:59.541525 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:59.541485    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:49:59.541709 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:59.541654    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 09:49:59.541709 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:59.541676    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-598bfb974b-kzqfn: secret "image-registry-tls" not found
Jul 30 09:49:59.541803 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:59.541744    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls podName:2f923caa-9667-4c0d-8083-47944d143e9d nodeName:}" failed. No retries permitted until 2026-07-30 09:50:01.541726567 +0000 UTC m=+38.035922051 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls") pod "image-registry-598bfb974b-kzqfn" (UID: "2f923caa-9667-4c0d-8083-47944d143e9d") : secret "image-registry-tls" not found
Jul 30 09:49:59.642806 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:49:59.642768    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:49:59.642989 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:59.642947    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 09:49:59.643039 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:49:59.643024    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls podName:3c1a1aad-170c-4e52-8f7d-e763663b81e4 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:01.643004106 +0000 UTC m=+38.137199612 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls") pod "dns-default-84b5h" (UID: "3c1a1aad-170c-4e52-8f7d-e763663b81e4") : secret "dns-default-metrics-tls" not found
Jul 30 09:50:00.301206 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.301114    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"]
Jul 30 09:50:00.304769 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.304749    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"
Jul 30 09:50:00.307120 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.307092    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 30 09:50:00.308054 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.308036    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 30 09:50:00.308142 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.308055    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-zpbgj\""
Jul 30 09:50:00.308142 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.308126    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 30 09:50:00.311612 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.311580    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"]
Jul 30 09:50:00.399446 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.399414    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-48xbr"]
Jul 30 09:50:00.402623 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.402599    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-48xbr"
Jul 30 09:50:00.405647 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.405485    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-tlj2z\""
Jul 30 09:50:00.405647 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.405506    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r"]
Jul 30 09:50:00.408429 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.408397    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"]
Jul 30 09:50:00.408825 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.408803    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r"
Jul 30 09:50:00.411436 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.411407    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-p8h62"]
Jul 30 09:50:00.411559 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.411538    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:00.411653 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.411640    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 30 09:50:00.412310 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.411868    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 30 09:50:00.412310 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.411937    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 30 09:50:00.412310 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.411869    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-n99z9\""
Jul 30 09:50:00.412310 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.412265    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 30 09:50:00.413800 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.413778    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 09:50:00.414752 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.414731    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 30 09:50:00.414853 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.414771    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ppvvv\""
Jul 30 09:50:00.415139 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.415120    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 09:50:00.415994 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.415913    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78"]
Jul 30 09:50:00.417206 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.416734    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.417206 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.416759    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 30 09:50:00.420760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.420404    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 30 09:50:00.420760 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.420602    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 09:50:00.420883 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.420817    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 09:50:00.421461 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.421443    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-48xbr"]
Jul 30 09:50:00.422080 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.421550    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78"
Jul 30 09:50:00.422627 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.422170    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r"]
Jul 30 09:50:00.422627 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.422412    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-6xgc8\""
Jul 30 09:50:00.422627 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.422462    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 30 09:50:00.423246 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.423224    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"]
Jul 30 09:50:00.423916 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.423897    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 30 09:50:00.424310 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.424291    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-p8h62"]
Jul 30 09:50:00.425253 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.424963    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5fbds\""
Jul 30 09:50:00.425253 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.425171    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 30 09:50:00.426249 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.425964    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 30 09:50:00.426249 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.426118    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 30 09:50:00.427913 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.427870    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 30 09:50:00.437352 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.437309    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78"]
Jul 30 09:50:00.450128 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.450073    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hjg9\" (UniqueName: \"kubernetes.io/projected/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-kube-api-access-4hjg9\") pod \"cluster-samples-operator-6748489656-bwpth\" (UID: \"b6d08841-c5eb-4c9e-a1cd-ffaaae91d021\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"
Jul 30 09:50:00.450128 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.450137    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bwpth\" (UID: \"b6d08841-c5eb-4c9e-a1cd-ffaaae91d021\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"
Jul 30 09:50:00.503168 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.503140    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-hw45c"]
Jul 30 09:50:00.506550 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.506529    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:00.509026 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.509008    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 30 09:50:00.509131 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.509111    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 30 09:50:00.509392 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.509359    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-8m484\""
Jul 30 09:50:00.509537 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.509519    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 30 09:50:00.509609 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.509558    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 30 09:50:00.510208 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.510188    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tbk9v"]
Jul 30 09:50:00.513186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.513165    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-888465b95-k24rp"]
Jul 30 09:50:00.513360 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.513323    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tbk9v"
Jul 30 09:50:00.515039 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.514978    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 30 09:50:00.516005 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.515989    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 30 09:50:00.516100 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.516068    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-wbdr5\""
Jul 30 09:50:00.516100 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.516080    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 30 09:50:00.516267 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.516249    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:00.518602 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.518584    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 30 09:50:00.518706 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.518640    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qzgp8\""
Jul 30 09:50:00.518706 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.518647    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 30 09:50:00.518858 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.518704    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 09:50:00.518858 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.518719    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 30 09:50:00.518858 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.518646    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 09:50:00.518858 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.518578    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 30 09:50:00.523827 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.523804    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-hw45c"]
Jul 30 09:50:00.529455 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.529422    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tbk9v"]
Jul 30 09:50:00.530787 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.530762    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-888465b95-k24rp"]
Jul 30 09:50:00.550635 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.550599    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/353dc841-0a51-4587-b4a9-6a4955ad1a12-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.550635 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.550635    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a79aad4c-afde-4ab7-b54a-4fd145a20252-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r\" (UID: \"a79aad4c-afde-4ab7-b54a-4fd145a20252\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r"
Jul 30 09:50:00.550999 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.550670    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5slhb\" (UniqueName: \"kubernetes.io/projected/a79aad4c-afde-4ab7-b54a-4fd145a20252-kube-api-access-5slhb\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r\" (UID: \"a79aad4c-afde-4ab7-b54a-4fd145a20252\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r"
Jul 30 09:50:00.550999 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.550690    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/353dc841-0a51-4587-b4a9-6a4955ad1a12-service-ca-bundle\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.550999 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.550732    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxvbw\" (UniqueName: \"kubernetes.io/projected/353dc841-0a51-4587-b4a9-6a4955ad1a12-kube-api-access-qxvbw\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.550999 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.550755    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/353dc841-0a51-4587-b4a9-6a4955ad1a12-tmp\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.550999 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.550783    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdwjl\" (UniqueName: \"kubernetes.io/projected/f9d4ea9c-d90d-4c74-bffb-d622bba6cafb-kube-api-access-xdwjl\") pod \"network-check-source-7d88cb55d8-48xbr\" (UID: \"f9d4ea9c-d90d-4c74-bffb-d622bba6cafb\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-48xbr"
Jul 30 09:50:00.550999 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.550810    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24fde817-36dc-436d-8706-47bdce301175-serving-cert\") pod \"service-ca-operator-5d6c75f84f-stc78\" (UID: \"24fde817-36dc-436d-8706-47bdce301175\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78"
Jul 30 09:50:00.550999 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.550855    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bwpth\" (UID: \"b6d08841-c5eb-4c9e-a1cd-ffaaae91d021\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"
Jul 30 09:50:00.550999 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.550903    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/da4c528d-7534-4727-8ace-474996ab3454-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:00.550999 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.550939    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8zdc\" (UniqueName: \"kubernetes.io/projected/24fde817-36dc-436d-8706-47bdce301175-kube-api-access-x8zdc\") pod \"service-ca-operator-5d6c75f84f-stc78\" (UID: \"24fde817-36dc-436d-8706-47bdce301175\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78"
Jul 30 09:50:00.550999 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:00.550958    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 09:50:00.551469 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.551012    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/353dc841-0a51-4587-b4a9-6a4955ad1a12-serving-cert\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.551469 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:00.551024    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls podName:b6d08841-c5eb-4c9e-a1cd-ffaaae91d021 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:01.051004743 +0000 UTC m=+37.545200248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls") pod "cluster-samples-operator-6748489656-bwpth" (UID: "b6d08841-c5eb-4c9e-a1cd-ffaaae91d021") : secret "samples-operator-tls" not found
Jul 30 09:50:00.551469 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.551071    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a79aad4c-afde-4ab7-b54a-4fd145a20252-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r\" (UID: \"a79aad4c-afde-4ab7-b54a-4fd145a20252\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r"
Jul 30 09:50:00.551469 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.551099    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/24fde817-36dc-436d-8706-47bdce301175-config\") pod \"service-ca-operator-5d6c75f84f-stc78\" (UID: \"24fde817-36dc-436d-8706-47bdce301175\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78"
Jul 30 09:50:00.551469 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.551128    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnjqb\" (UniqueName: \"kubernetes.io/projected/da4c528d-7534-4727-8ace-474996ab3454-kube-api-access-hnjqb\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:00.551469 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.551164    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/353dc841-0a51-4587-b4a9-6a4955ad1a12-snapshots\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.551469 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.551192    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hjg9\" (UniqueName: \"kubernetes.io/projected/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-kube-api-access-4hjg9\") pod \"cluster-samples-operator-6748489656-bwpth\" (UID: \"b6d08841-c5eb-4c9e-a1cd-ffaaae91d021\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"
Jul 30 09:50:00.551469 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.551219    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:00.561770 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.561737    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hjg9\" (UniqueName: \"kubernetes.io/projected/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-kube-api-access-4hjg9\") pod \"cluster-samples-operator-6748489656-bwpth\" (UID: \"b6d08841-c5eb-4c9e-a1cd-ffaaae91d021\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"
Jul 30 09:50:00.644809 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.644772    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log" limit=1024
Jul 30 09:50:00.651571 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.651545    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdwjl\" (UniqueName: \"kubernetes.io/projected/f9d4ea9c-d90d-4c74-bffb-d622bba6cafb-kube-api-access-xdwjl\") pod \"network-check-source-7d88cb55d8-48xbr\" (UID: \"f9d4ea9c-d90d-4c74-bffb-d622bba6cafb\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-48xbr"
Jul 30 09:50:00.651741 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.651579    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24fde817-36dc-436d-8706-47bdce301175-serving-cert\") pod \"service-ca-operator-5d6c75f84f-stc78\" (UID: \"24fde817-36dc-436d-8706-47bdce301175\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78"
Jul 30 09:50:00.651741 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.651607    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:00.651860 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.651737    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-stats-auth\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:00.651860 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.651810    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/97e78a90-a448-4e2d-bdf4-5b21892bb617-config\") pod \"console-operator-584dcb6b54-hw45c\" (UID: \"97e78a90-a448-4e2d-bdf4-5b21892bb617\") " pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:00.651860 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.651841    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:00.652003 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.651874    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/da4c528d-7534-4727-8ace-474996ab3454-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:00.652003 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.651903    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8zdc\" (UniqueName: \"kubernetes.io/projected/24fde817-36dc-436d-8706-47bdce301175-kube-api-access-x8zdc\") pod \"service-ca-operator-5d6c75f84f-stc78\" (UID: \"24fde817-36dc-436d-8706-47bdce301175\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78"
Jul 30 09:50:00.652118 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652086    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/353dc841-0a51-4587-b4a9-6a4955ad1a12-serving-cert\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.652173 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652137    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkl6p\" (UniqueName: \"kubernetes.io/projected/97e78a90-a448-4e2d-bdf4-5b21892bb617-kube-api-access-pkl6p\") pod \"console-operator-584dcb6b54-hw45c\" (UID: \"97e78a90-a448-4e2d-bdf4-5b21892bb617\") " pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:00.652226 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652188    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a79aad4c-afde-4ab7-b54a-4fd145a20252-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r\" (UID: \"a79aad4c-afde-4ab7-b54a-4fd145a20252\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r"
Jul 30 09:50:00.652226 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652218    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/24fde817-36dc-436d-8706-47bdce301175-config\") pod \"service-ca-operator-5d6c75f84f-stc78\" (UID: \"24fde817-36dc-436d-8706-47bdce301175\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78"
Jul 30 09:50:00.652300 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652248    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqkct\" (UniqueName: \"kubernetes.io/projected/e5551d48-3f70-4f9a-80eb-9daf2a25cf6d-kube-api-access-mqkct\") pod \"volume-data-source-validator-549c6d54cf-tbk9v\" (UID: \"e5551d48-3f70-4f9a-80eb-9daf2a25cf6d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tbk9v"
Jul 30 09:50:00.652300 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652281    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnjqb\" (UniqueName: \"kubernetes.io/projected/da4c528d-7534-4727-8ace-474996ab3454-kube-api-access-hnjqb\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:00.652386 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652323    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/353dc841-0a51-4587-b4a9-6a4955ad1a12-snapshots\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.652386 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652378    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:00.652539 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652427    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/353dc841-0a51-4587-b4a9-6a4955ad1a12-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.652539 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652455    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-default-certificate\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:00.652539 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652491    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a79aad4c-afde-4ab7-b54a-4fd145a20252-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r\" (UID: \"a79aad4c-afde-4ab7-b54a-4fd145a20252\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r"
Jul 30 09:50:00.652539 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652524    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/97e78a90-a448-4e2d-bdf4-5b21892bb617-serving-cert\") pod \"console-operator-584dcb6b54-hw45c\" (UID: \"97e78a90-a448-4e2d-bdf4-5b21892bb617\") " pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:00.652736 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652552    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97e78a90-a448-4e2d-bdf4-5b21892bb617-trusted-ca\") pod \"console-operator-584dcb6b54-hw45c\" (UID: \"97e78a90-a448-4e2d-bdf4-5b21892bb617\") " pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:00.652736 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652579    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5slhb\" (UniqueName: \"kubernetes.io/projected/a79aad4c-afde-4ab7-b54a-4fd145a20252-kube-api-access-5slhb\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r\" (UID: \"a79aad4c-afde-4ab7-b54a-4fd145a20252\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r"
Jul 30 09:50:00.652736 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652603    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/353dc841-0a51-4587-b4a9-6a4955ad1a12-service-ca-bundle\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.652736 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652628    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxvbw\" (UniqueName: \"kubernetes.io/projected/353dc841-0a51-4587-b4a9-6a4955ad1a12-kube-api-access-qxvbw\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.652736 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652659    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/353dc841-0a51-4587-b4a9-6a4955ad1a12-tmp\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.652736 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652689    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghdjn\" (UniqueName: \"kubernetes.io/projected/b19ed290-2519-455d-8147-688ac9ed11b0-kube-api-access-ghdjn\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:00.652736 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652698    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/da4c528d-7534-4727-8ace-474996ab3454-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:00.653064 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:00.652791    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 09:50:00.653064 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:00.652852    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls podName:da4c528d-7534-4727-8ace-474996ab3454 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:01.152835483 +0000 UTC m=+37.647030983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-mz9gf" (UID: "da4c528d-7534-4727-8ace-474996ab3454") : secret "cluster-monitoring-operator-tls" not found
Jul 30 09:50:00.653064 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.652957    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/24fde817-36dc-436d-8706-47bdce301175-config\") pod \"service-ca-operator-5d6c75f84f-stc78\" (UID: \"24fde817-36dc-436d-8706-47bdce301175\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78"
Jul 30 09:50:00.653227 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.653114    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/353dc841-0a51-4587-b4a9-6a4955ad1a12-snapshots\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.653586 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.653548    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a79aad4c-afde-4ab7-b54a-4fd145a20252-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r\" (UID: \"a79aad4c-afde-4ab7-b54a-4fd145a20252\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r"
Jul 30 09:50:00.653711 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.653683    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/353dc841-0a51-4587-b4a9-6a4955ad1a12-tmp\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.653811 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.653791    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/353dc841-0a51-4587-b4a9-6a4955ad1a12-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.654035 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.654012    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/353dc841-0a51-4587-b4a9-6a4955ad1a12-service-ca-bundle\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.654882 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.654840    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a79aad4c-afde-4ab7-b54a-4fd145a20252-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r\" (UID: \"a79aad4c-afde-4ab7-b54a-4fd145a20252\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r"
Jul 30 09:50:00.654882 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.654871    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/353dc841-0a51-4587-b4a9-6a4955ad1a12-serving-cert\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.655045 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.655008    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24fde817-36dc-436d-8706-47bdce301175-serving-cert\") pod \"service-ca-operator-5d6c75f84f-stc78\" (UID: \"24fde817-36dc-436d-8706-47bdce301175\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78"
Jul 30 09:50:00.660092 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.660065    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdwjl\" (UniqueName: \"kubernetes.io/projected/f9d4ea9c-d90d-4c74-bffb-d622bba6cafb-kube-api-access-xdwjl\") pod \"network-check-source-7d88cb55d8-48xbr\" (UID: \"f9d4ea9c-d90d-4c74-bffb-d622bba6cafb\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-48xbr"
Jul 30 09:50:00.660231 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.660163    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8zdc\" (UniqueName: \"kubernetes.io/projected/24fde817-36dc-436d-8706-47bdce301175-kube-api-access-x8zdc\") pod \"service-ca-operator-5d6c75f84f-stc78\" (UID: \"24fde817-36dc-436d-8706-47bdce301175\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78"
Jul 30 09:50:00.662597 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.662576    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5slhb\" (UniqueName: \"kubernetes.io/projected/a79aad4c-afde-4ab7-b54a-4fd145a20252-kube-api-access-5slhb\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r\" (UID: \"a79aad4c-afde-4ab7-b54a-4fd145a20252\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r"
Jul 30 09:50:00.662685 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.662575    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnjqb\" (UniqueName: \"kubernetes.io/projected/da4c528d-7534-4727-8ace-474996ab3454-kube-api-access-hnjqb\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:00.662738 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.662703    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxvbw\" (UniqueName: \"kubernetes.io/projected/353dc841-0a51-4587-b4a9-6a4955ad1a12-kube-api-access-qxvbw\") pod \"insights-operator-5885fdb759-p8h62\" (UID: \"353dc841-0a51-4587-b4a9-6a4955ad1a12\") " pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.745866 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.745776    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log" limit=1024
Jul 30 09:50:00.753724 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.753701    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/97e78a90-a448-4e2d-bdf4-5b21892bb617-config\") pod \"console-operator-584dcb6b54-hw45c\" (UID: \"97e78a90-a448-4e2d-bdf4-5b21892bb617\") " pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:00.753822 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.753730    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:00.753822 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.753779    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkl6p\" (UniqueName: \"kubernetes.io/projected/97e78a90-a448-4e2d-bdf4-5b21892bb617-kube-api-access-pkl6p\") pod \"console-operator-584dcb6b54-hw45c\" (UID: \"97e78a90-a448-4e2d-bdf4-5b21892bb617\") " pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:00.753913 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.753824    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqkct\" (UniqueName: \"kubernetes.io/projected/e5551d48-3f70-4f9a-80eb-9daf2a25cf6d-kube-api-access-mqkct\") pod \"volume-data-source-validator-549c6d54cf-tbk9v\" (UID: \"e5551d48-3f70-4f9a-80eb-9daf2a25cf6d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tbk9v"
Jul 30 09:50:00.753913 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:00.753883    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle podName:b19ed290-2519-455d-8147-688ac9ed11b0 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:01.253866693 +0000 UTC m=+37.748062178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle") pod "router-default-888465b95-k24rp" (UID: "b19ed290-2519-455d-8147-688ac9ed11b0") : configmap references non-existent config key: service-ca.crt
Jul 30 09:50:00.754016 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.753929    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-default-certificate\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:00.754016 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.753955    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/97e78a90-a448-4e2d-bdf4-5b21892bb617-serving-cert\") pod \"console-operator-584dcb6b54-hw45c\" (UID: \"97e78a90-a448-4e2d-bdf4-5b21892bb617\") " pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:00.754016 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.753982    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97e78a90-a448-4e2d-bdf4-5b21892bb617-trusted-ca\") pod \"console-operator-584dcb6b54-hw45c\" (UID: \"97e78a90-a448-4e2d-bdf4-5b21892bb617\") " pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:00.754161 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.754027    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghdjn\" (UniqueName: \"kubernetes.io/projected/b19ed290-2519-455d-8147-688ac9ed11b0-kube-api-access-ghdjn\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:00.754161 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.754057    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:00.754161 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.754086    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-stats-auth\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:00.754404 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:00.754252    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 09:50:00.754404 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:00.754315    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs podName:b19ed290-2519-455d-8147-688ac9ed11b0 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:01.254298837 +0000 UTC m=+37.748494324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs") pod "router-default-888465b95-k24rp" (UID: "b19ed290-2519-455d-8147-688ac9ed11b0") : secret "router-metrics-certs-default" not found
Jul 30 09:50:00.754563 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.754540    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/97e78a90-a448-4e2d-bdf4-5b21892bb617-config\") pod \"console-operator-584dcb6b54-hw45c\" (UID: \"97e78a90-a448-4e2d-bdf4-5b21892bb617\") " pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:00.754858 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.754836    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97e78a90-a448-4e2d-bdf4-5b21892bb617-trusted-ca\") pod \"console-operator-584dcb6b54-hw45c\" (UID: \"97e78a90-a448-4e2d-bdf4-5b21892bb617\") " pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:00.756500 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.756479    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/97e78a90-a448-4e2d-bdf4-5b21892bb617-serving-cert\") pod \"console-operator-584dcb6b54-hw45c\" (UID: \"97e78a90-a448-4e2d-bdf4-5b21892bb617\") " pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:00.756588 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.756513    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-stats-auth\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:00.756719 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.756699    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-default-certificate\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:00.762907 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.762879    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghdjn\" (UniqueName: \"kubernetes.io/projected/b19ed290-2519-455d-8147-688ac9ed11b0-kube-api-access-ghdjn\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:00.763037 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.763017    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkl6p\" (UniqueName: \"kubernetes.io/projected/97e78a90-a448-4e2d-bdf4-5b21892bb617-kube-api-access-pkl6p\") pod \"console-operator-584dcb6b54-hw45c\" (UID: \"97e78a90-a448-4e2d-bdf4-5b21892bb617\") " pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:00.763108 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.763092    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqkct\" (UniqueName: \"kubernetes.io/projected/e5551d48-3f70-4f9a-80eb-9daf2a25cf6d-kube-api-access-mqkct\") pod \"volume-data-source-validator-549c6d54cf-tbk9v\" (UID: \"e5551d48-3f70-4f9a-80eb-9daf2a25cf6d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tbk9v"
Jul 30 09:50:00.778163 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.778135    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-48xbr"
Jul 30 09:50:00.781446 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.781415    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r"
Jul 30 09:50:00.796062 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.795842    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-p8h62"
Jul 30 09:50:00.802728 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.802541    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78"
Jul 30 09:50:00.818475 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.818385    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:00.828173 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:00.828065    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tbk9v"
Jul 30 09:50:01.024537 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.024494    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r"]
Jul 30 09:50:01.031388 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:01.031104    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda79aad4c_afde_4ab7_b54a_4fd145a20252.slice/crio-496b8eaf09a345e83b4a7e228cbb628f0e15226b0a8ba8e574f6a3c35be1f293 WatchSource:0}: Error finding container 496b8eaf09a345e83b4a7e228cbb628f0e15226b0a8ba8e574f6a3c35be1f293: Status 404 returned error can't find the container with id 496b8eaf09a345e83b4a7e228cbb628f0e15226b0a8ba8e574f6a3c35be1f293
Jul 30 09:50:01.063652 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.063611    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-48xbr"]
Jul 30 09:50:01.067223 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.067194    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bwpth\" (UID: \"b6d08841-c5eb-4c9e-a1cd-ffaaae91d021\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"
Jul 30 09:50:01.067379 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:01.067356    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 09:50:01.067450 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:01.067425    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls podName:b6d08841-c5eb-4c9e-a1cd-ffaaae91d021 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:02.067403943 +0000 UTC m=+38.561599430 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls") pod "cluster-samples-operator-6748489656-bwpth" (UID: "b6d08841-c5eb-4c9e-a1cd-ffaaae91d021") : secret "samples-operator-tls" not found
Jul 30 09:50:01.067994 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:01.067878    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d4ea9c_d90d_4c74_bffb_d622bba6cafb.slice/crio-7c7a9d3ad6ab5c387bbf1027c9a4850bcc9f50e18e43a8949904c4f1e00e5765 WatchSource:0}: Error finding container 7c7a9d3ad6ab5c387bbf1027c9a4850bcc9f50e18e43a8949904c4f1e00e5765: Status 404 returned error can't find the container with id 7c7a9d3ad6ab5c387bbf1027c9a4850bcc9f50e18e43a8949904c4f1e00e5765
Jul 30 09:50:01.068765 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.068669    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78"]
Jul 30 09:50:01.071317 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:01.071288    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24fde817_36dc_436d_8706_47bdce301175.slice/crio-bb98a51a993d487b2859a4d2107852292aae28dd207a0c1c3503ea73dfea6350 WatchSource:0}: Error finding container bb98a51a993d487b2859a4d2107852292aae28dd207a0c1c3503ea73dfea6350: Status 404 returned error can't find the container with id bb98a51a993d487b2859a4d2107852292aae28dd207a0c1c3503ea73dfea6350
Jul 30 09:50:01.081397 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.081370    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tbk9v"]
Jul 30 09:50:01.085131 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:01.085098    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5551d48_3f70_4f9a_80eb_9daf2a25cf6d.slice/crio-6d9da15061803af39a23540998ff040c82e16b45e9105bf7d84ce5b38d0af247 WatchSource:0}: Error finding container 6d9da15061803af39a23540998ff040c82e16b45e9105bf7d84ce5b38d0af247: Status 404 returned error can't find the container with id 6d9da15061803af39a23540998ff040c82e16b45e9105bf7d84ce5b38d0af247
Jul 30 09:50:01.085737 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.085681    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-p8h62"]
Jul 30 09:50:01.088918 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:01.088895    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353dc841_0a51_4587_b4a9_6a4955ad1a12.slice/crio-35a2565b04e4b33e749fa26d2ffa4be639482f2421ae97693d62657805ab7028 WatchSource:0}: Error finding container 35a2565b04e4b33e749fa26d2ffa4be639482f2421ae97693d62657805ab7028: Status 404 returned error can't find the container with id 35a2565b04e4b33e749fa26d2ffa4be639482f2421ae97693d62657805ab7028
Jul 30 09:50:01.101614 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.101483    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-hw45c"]
Jul 30 09:50:01.104042 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:01.104006    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e78a90_a448_4e2d_bdf4_5b21892bb617.slice/crio-dd5b658aa55c21ea621a4378c847f27c9c968a4082e47dd43cf370262364d957 WatchSource:0}: Error finding container dd5b658aa55c21ea621a4378c847f27c9c968a4082e47dd43cf370262364d957: Status 404 returned error can't find the container with id dd5b658aa55c21ea621a4378c847f27c9c968a4082e47dd43cf370262364d957
Jul 30 09:50:01.167963 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.167924    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:01.168121 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:01.168065    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 09:50:01.168167 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:01.168133    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls podName:da4c528d-7534-4727-8ace-474996ab3454 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:02.168117453 +0000 UTC m=+38.662312938 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-mz9gf" (UID: "da4c528d-7534-4727-8ace-474996ab3454") : secret "cluster-monitoring-operator-tls" not found
Jul 30 09:50:01.269138 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.269062    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:01.269138 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.269131    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:01.269414 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:01.269320    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 09:50:01.269414 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:01.269374    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle podName:b19ed290-2519-455d-8147-688ac9ed11b0 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:02.269326177 +0000 UTC m=+38.763521679 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle") pod "router-default-888465b95-k24rp" (UID: "b19ed290-2519-455d-8147-688ac9ed11b0") : configmap references non-existent config key: service-ca.crt
Jul 30 09:50:01.269414 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:01.269407    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs podName:b19ed290-2519-455d-8147-688ac9ed11b0 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:02.269390507 +0000 UTC m=+38.763585994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs") pod "router-default-888465b95-k24rp" (UID: "b19ed290-2519-455d-8147-688ac9ed11b0") : secret "router-metrics-certs-default" not found
Jul 30 09:50:01.272681 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.272632    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78" event={"ID":"24fde817-36dc-436d-8706-47bdce301175","Type":"ContainerStarted","Data":"bb98a51a993d487b2859a4d2107852292aae28dd207a0c1c3503ea73dfea6350"}
Jul 30 09:50:01.274415 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.274331    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-p8h62" event={"ID":"353dc841-0a51-4587-b4a9-6a4955ad1a12","Type":"ContainerStarted","Data":"35a2565b04e4b33e749fa26d2ffa4be639482f2421ae97693d62657805ab7028"}
Jul 30 09:50:01.276354 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.276301    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-48xbr" event={"ID":"f9d4ea9c-d90d-4c74-bffb-d622bba6cafb","Type":"ContainerStarted","Data":"7c7a9d3ad6ab5c387bbf1027c9a4850bcc9f50e18e43a8949904c4f1e00e5765"}
Jul 30 09:50:01.279845 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.279800    2567 generic.go:356] "Generic (PLEG): container finished" podID="e69de26a-c120-4b24-a283-04e2db290ed4" containerID="e25af48058b4bdac6007addb8a1a7cbb5aabf62a99a0cbcff21d731b0eec5f31" exitCode=0
Jul 30 09:50:01.279952 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.279889    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98kcx" event={"ID":"e69de26a-c120-4b24-a283-04e2db290ed4","Type":"ContainerDied","Data":"e25af48058b4bdac6007addb8a1a7cbb5aabf62a99a0cbcff21d731b0eec5f31"}
Jul 30 09:50:01.281226 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.281071    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c" event={"ID":"97e78a90-a448-4e2d-bdf4-5b21892bb617","Type":"ContainerStarted","Data":"dd5b658aa55c21ea621a4378c847f27c9c968a4082e47dd43cf370262364d957"}
Jul 30 09:50:01.282216 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.282186    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r" event={"ID":"a79aad4c-afde-4ab7-b54a-4fd145a20252","Type":"ContainerStarted","Data":"496b8eaf09a345e83b4a7e228cbb628f0e15226b0a8ba8e574f6a3c35be1f293"}
Jul 30 09:50:01.283514 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.283491    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tbk9v" event={"ID":"e5551d48-3f70-4f9a-80eb-9daf2a25cf6d","Type":"ContainerStarted","Data":"6d9da15061803af39a23540998ff040c82e16b45e9105bf7d84ce5b38d0af247"}
Jul 30 09:50:01.572425 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.572324    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:50:01.572900 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:01.572469    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 09:50:01.572900 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:01.572494    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-598bfb974b-kzqfn: secret "image-registry-tls" not found
Jul 30 09:50:01.572900 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:01.572564    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls podName:2f923caa-9667-4c0d-8083-47944d143e9d nodeName:}" failed. No retries permitted until 2026-07-30 09:50:05.572542806 +0000 UTC m=+42.066738305 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls") pod "image-registry-598bfb974b-kzqfn" (UID: "2f923caa-9667-4c0d-8083-47944d143e9d") : secret "image-registry-tls" not found
Jul 30 09:50:01.673392 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:01.673326    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:50:01.673536 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:01.673483    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 09:50:01.673571 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:01.673554    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls podName:3c1a1aad-170c-4e52-8f7d-e763663b81e4 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:05.673534169 +0000 UTC m=+42.167729658 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls") pod "dns-default-84b5h" (UID: "3c1a1aad-170c-4e52-8f7d-e763663b81e4") : secret "dns-default-metrics-tls" not found
Jul 30 09:50:02.076644 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:02.076605    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bwpth\" (UID: \"b6d08841-c5eb-4c9e-a1cd-ffaaae91d021\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"
Jul 30 09:50:02.076867 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:02.076849    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 09:50:02.076931 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:02.076919    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls podName:b6d08841-c5eb-4c9e-a1cd-ffaaae91d021 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:04.076898322 +0000 UTC m=+40.571093806 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls") pod "cluster-samples-operator-6748489656-bwpth" (UID: "b6d08841-c5eb-4c9e-a1cd-ffaaae91d021") : secret "samples-operator-tls" not found
Jul 30 09:50:02.178486 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:02.177891    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:02.178486 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:02.178043    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 09:50:02.178486 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:02.178113    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls podName:da4c528d-7534-4727-8ace-474996ab3454 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:04.178091729 +0000 UTC m=+40.672287216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-mz9gf" (UID: "da4c528d-7534-4727-8ace-474996ab3454") : secret "cluster-monitoring-operator-tls" not found
Jul 30 09:50:02.279044 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:02.279003    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:02.279235 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:02.279083    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:02.279304 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:02.279262    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 09:50:02.279371 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:02.279358    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle podName:b19ed290-2519-455d-8147-688ac9ed11b0 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:04.279321037 +0000 UTC m=+40.773516539 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle") pod "router-default-888465b95-k24rp" (UID: "b19ed290-2519-455d-8147-688ac9ed11b0") : configmap references non-existent config key: service-ca.crt
Jul 30 09:50:02.279432 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:02.279384    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs podName:b19ed290-2519-455d-8147-688ac9ed11b0 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:04.279372271 +0000 UTC m=+40.773567763 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs") pod "router-default-888465b95-k24rp" (UID: "b19ed290-2519-455d-8147-688ac9ed11b0") : secret "router-metrics-certs-default" not found
Jul 30 09:50:02.291625 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:02.291590    2567 generic.go:356] "Generic (PLEG): container finished" podID="e69de26a-c120-4b24-a283-04e2db290ed4" containerID="5b4f73fcd1ef30ae0bba43db3d88453c137dc51ea43b6154a9deaee30f78c8a2" exitCode=0
Jul 30 09:50:02.291784 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:02.291635    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98kcx" event={"ID":"e69de26a-c120-4b24-a283-04e2db290ed4","Type":"ContainerDied","Data":"5b4f73fcd1ef30ae0bba43db3d88453c137dc51ea43b6154a9deaee30f78c8a2"}
Jul 30 09:50:02.548571 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:02.548533    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log" limit=1024
Jul 30 09:50:02.619778 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:02.619710    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log" limit=1024
Jul 30 09:50:03.089452 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.089413    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"]
Jul 30 09:50:03.110942 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.110909    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"]
Jul 30 09:50:03.111115 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.111048    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"
Jul 30 09:50:03.113558 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.113526    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 30 09:50:03.113701 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.113685    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-k54mz\""
Jul 30 09:50:03.113777 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.113702    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 30 09:50:03.218203 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.218165    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log" limit=1024
Jul 30 09:50:03.288093 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.287999    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8d349f02-e060-4dca-b9b9-44163a0fb935-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-fhkxr\" (UID: \"8d349f02-e060-4dca-b9b9-44163a0fb935\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"
Jul 30 09:50:03.288255 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.288091    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fhkxr\" (UID: \"8d349f02-e060-4dca-b9b9-44163a0fb935\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"
Jul 30 09:50:03.297857 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.297826    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-98kcx" event={"ID":"e69de26a-c120-4b24-a283-04e2db290ed4","Type":"ContainerStarted","Data":"de86ada0dfe0b6f5142ba3228c8d8eb2da9971d4feb19c4f8e773111c6bfcd1f"}
Jul 30 09:50:03.315954 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.315917    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log" limit=1024
Jul 30 09:50:03.322081 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.322024    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-98kcx" podStartSLOduration=8.539870377 podStartE2EDuration="39.322003309s" totalImagesPullingTime="30.782132932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:03.320663046 +0000 UTC m=+39.814858551" watchObservedRunningTime="2026-07-30 09:50:03.322003309 +0000 UTC m=+39.816198816"
Jul 30 09:50:03.389289 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.389246    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fhkxr\" (UID: \"8d349f02-e060-4dca-b9b9-44163a0fb935\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"
Jul 30 09:50:03.389503 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.389421    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8d349f02-e060-4dca-b9b9-44163a0fb935-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-fhkxr\" (UID: \"8d349f02-e060-4dca-b9b9-44163a0fb935\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"
Jul 30 09:50:03.389503 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:03.389460    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 09:50:03.389625 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:03.389537    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert podName:8d349f02-e060-4dca-b9b9-44163a0fb935 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:03.889515687 +0000 UTC m=+40.383711197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-fhkxr" (UID: "8d349f02-e060-4dca-b9b9-44163a0fb935") : secret "networking-console-plugin-cert" not found
Jul 30 09:50:03.390189 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.390152    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8d349f02-e060-4dca-b9b9-44163a0fb935-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-fhkxr\" (UID: \"8d349f02-e060-4dca-b9b9-44163a0fb935\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"
Jul 30 09:50:03.418080 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.418040    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log" limit=1024
Jul 30 09:50:03.481768 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.481628    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log" limit=1024
Jul 30 09:50:03.894635 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:03.894550    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fhkxr\" (UID: \"8d349f02-e060-4dca-b9b9-44163a0fb935\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"
Jul 30 09:50:03.895177 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:03.894685    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 09:50:03.895177 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:03.894767    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert podName:8d349f02-e060-4dca-b9b9-44163a0fb935 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:04.894747659 +0000 UTC m=+41.388943145 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-fhkxr" (UID: "8d349f02-e060-4dca-b9b9-44163a0fb935") : secret "networking-console-plugin-cert" not found
Jul 30 09:50:04.096486 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:04.096396    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bwpth\" (UID: \"b6d08841-c5eb-4c9e-a1cd-ffaaae91d021\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"
Jul 30 09:50:04.096661 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:04.096580    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 09:50:04.096776 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:04.096662    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls podName:b6d08841-c5eb-4c9e-a1cd-ffaaae91d021 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:08.096637394 +0000 UTC m=+44.590832898 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls") pod "cluster-samples-operator-6748489656-bwpth" (UID: "b6d08841-c5eb-4c9e-a1cd-ffaaae91d021") : secret "samples-operator-tls" not found
Jul 30 09:50:04.162628 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:04.162497    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log" limit=1024
Jul 30 09:50:04.198030 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:04.197955    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:04.198221 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:04.198092    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 09:50:04.198221 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:04.198174    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls podName:da4c528d-7534-4727-8ace-474996ab3454 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:08.198149258 +0000 UTC m=+44.692344745 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-mz9gf" (UID: "da4c528d-7534-4727-8ace-474996ab3454") : secret "cluster-monitoring-operator-tls" not found
Jul 30 09:50:04.264251 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:04.264146    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log" limit=1024
Jul 30 09:50:04.299092 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:04.299055    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:04.299285 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:04.299128    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:04.299285 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:04.299219    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 09:50:04.299632 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:04.299289    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs podName:b19ed290-2519-455d-8147-688ac9ed11b0 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:08.29927066 +0000 UTC m=+44.793466151 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs") pod "router-default-888465b95-k24rp" (UID: "b19ed290-2519-455d-8147-688ac9ed11b0") : secret "router-metrics-certs-default" not found
Jul 30 09:50:04.299632 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:04.299349    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle podName:b19ed290-2519-455d-8147-688ac9ed11b0 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:08.299315927 +0000 UTC m=+44.793511417 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle") pod "router-default-888465b95-k24rp" (UID: "b19ed290-2519-455d-8147-688ac9ed11b0") : configmap references non-existent config key: service-ca.crt
Jul 30 09:50:04.567681 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:04.567601    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log" limit=1024
Jul 30 09:50:04.636079 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:04.636042    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log" limit=1024
Jul 30 09:50:04.905906 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:04.905871    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fhkxr\" (UID: \"8d349f02-e060-4dca-b9b9-44163a0fb935\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"
Jul 30 09:50:04.906383 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:04.906026    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 09:50:04.906383 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:04.906095    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert podName:8d349f02-e060-4dca-b9b9-44163a0fb935 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:06.906078952 +0000 UTC m=+43.400274442 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-fhkxr" (UID: "8d349f02-e060-4dca-b9b9-44163a0fb935") : secret "networking-console-plugin-cert" not found
Jul 30 09:50:05.612813 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:05.612777    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:50:05.612998 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:05.612953    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 09:50:05.612998 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:05.612973    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-598bfb974b-kzqfn: secret "image-registry-tls" not found
Jul 30 09:50:05.613098 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:05.613030    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls podName:2f923caa-9667-4c0d-8083-47944d143e9d nodeName:}" failed. No retries permitted until 2026-07-30 09:50:13.613010537 +0000 UTC m=+50.107206020 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls") pod "image-registry-598bfb974b-kzqfn" (UID: "2f923caa-9667-4c0d-8083-47944d143e9d") : secret "image-registry-tls" not found
Jul 30 09:50:05.714179 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:05.714135    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:50:05.714380 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:05.714307    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 09:50:05.714461 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:05.714407    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls podName:3c1a1aad-170c-4e52-8f7d-e763663b81e4 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:13.714386503 +0000 UTC m=+50.208581996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls") pod "dns-default-84b5h" (UID: "3c1a1aad-170c-4e52-8f7d-e763663b81e4") : secret "dns-default-metrics-tls" not found
Jul 30 09:50:06.926664 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:06.926620    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fhkxr\" (UID: \"8d349f02-e060-4dca-b9b9-44163a0fb935\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"
Jul 30 09:50:06.927077 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:06.926802    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 09:50:06.927077 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:06.926883    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert podName:8d349f02-e060-4dca-b9b9-44163a0fb935 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:10.926860882 +0000 UTC m=+47.421056369 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-fhkxr" (UID: "8d349f02-e060-4dca-b9b9-44163a0fb935") : secret "networking-console-plugin-cert" not found
Jul 30 09:50:08.139244 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:08.139207    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bwpth\" (UID: \"b6d08841-c5eb-4c9e-a1cd-ffaaae91d021\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"
Jul 30 09:50:08.139640 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:08.139364    2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 30 09:50:08.139640 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:08.139428    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls podName:b6d08841-c5eb-4c9e-a1cd-ffaaae91d021 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:16.139413304 +0000 UTC m=+52.633608787 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls") pod "cluster-samples-operator-6748489656-bwpth" (UID: "b6d08841-c5eb-4c9e-a1cd-ffaaae91d021") : secret "samples-operator-tls" not found
Jul 30 09:50:08.239995 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:08.239898    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:08.240142 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:08.240048    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 09:50:08.240142 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:08.240113    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls podName:da4c528d-7534-4727-8ace-474996ab3454 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:16.240097136 +0000 UTC m=+52.734292620 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-mz9gf" (UID: "da4c528d-7534-4727-8ace-474996ab3454") : secret "cluster-monitoring-operator-tls" not found
Jul 30 09:50:08.341130 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:08.341103    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:08.341278 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:08.341164    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:08.341362 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:08.341283    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 09:50:08.341362 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:08.341320    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle podName:b19ed290-2519-455d-8147-688ac9ed11b0 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:16.341300358 +0000 UTC m=+52.835495862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle") pod "router-default-888465b95-k24rp" (UID: "b19ed290-2519-455d-8147-688ac9ed11b0") : configmap references non-existent config key: service-ca.crt
Jul 30 09:50:08.341476 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:08.341370    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs podName:b19ed290-2519-455d-8147-688ac9ed11b0 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:16.34132982 +0000 UTC m=+52.835525321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs") pod "router-default-888465b95-k24rp" (UID: "b19ed290-2519-455d-8147-688ac9ed11b0") : secret "router-metrics-certs-default" not found
Jul 30 09:50:09.315998 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.315971    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/0.log"
Jul 30 09:50:09.316458 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.316015    2567 generic.go:356] "Generic (PLEG): container finished" podID="97e78a90-a448-4e2d-bdf4-5b21892bb617" containerID="d96583199d671da41957940613087e6af9221cbee223c29b732c4ddf43afddc7" exitCode=255
Jul 30 09:50:09.316458 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.316095    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c" event={"ID":"97e78a90-a448-4e2d-bdf4-5b21892bb617","Type":"ContainerDied","Data":"d96583199d671da41957940613087e6af9221cbee223c29b732c4ddf43afddc7"}
Jul 30 09:50:09.316458 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.316425    2567 scope.go:117] "RemoveContainer" containerID="d96583199d671da41957940613087e6af9221cbee223c29b732c4ddf43afddc7"
Jul 30 09:50:09.318259 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.318231    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r" event={"ID":"a79aad4c-afde-4ab7-b54a-4fd145a20252","Type":"ContainerStarted","Data":"314b34a3a7e94d78896ee1b5e66cca094028cdd162765b44d9daf8e6f20be3a4"}
Jul 30 09:50:09.320097 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.320066    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tbk9v" event={"ID":"e5551d48-3f70-4f9a-80eb-9daf2a25cf6d","Type":"ContainerStarted","Data":"e981e9ad0cd50f8e8b75ed1e088be532d7cc9cb729613aaba6a04ee1a7ff6b6a"}
Jul 30 09:50:09.321722 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.321698    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78" event={"ID":"24fde817-36dc-436d-8706-47bdce301175","Type":"ContainerStarted","Data":"93dfd7be50a1b0a25fa879744304d404edcd6cb014c76a3bafca9c25a9715d79"}
Jul 30 09:50:09.323483 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.323460    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zhg8c" event={"ID":"c89418e7-06ea-4bb2-8c58-22fbd57034b5","Type":"ContainerStarted","Data":"411a525c7d3d73a04b18ef33a8e7dd27339ab951177917c8216faae1e5fac4aa"}
Jul 30 09:50:09.325462 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.325161    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-p8h62" event={"ID":"353dc841-0a51-4587-b4a9-6a4955ad1a12","Type":"ContainerStarted","Data":"c5646de131751c7fd7fe8c0132ecf144a5a93ba0eb17732540a99757d7493ae0"}
Jul 30 09:50:09.326795 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.326773    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-48xbr" event={"ID":"f9d4ea9c-d90d-4c74-bffb-d622bba6cafb","Type":"ContainerStarted","Data":"dd19ce42338eea255260f045ccec15f1debab1bdee9b3938b01b632b23e3cefd"}
Jul 30 09:50:09.361708 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.361564    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-48xbr" podStartSLOduration=2.16078041 podStartE2EDuration="9.361545856s" totalImagesPullingTime="7.200765446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:09.355186761 +0000 UTC m=+45.849382268" watchObservedRunningTime="2026-07-30 09:50:09.361545856 +0000 UTC m=+45.855741365"
Jul 30 09:50:09.395925 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.395572    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-p8h62" podStartSLOduration=2.17896992 podStartE2EDuration="9.395550274s" totalImagesPullingTime="7.216580354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:09.377510979 +0000 UTC m=+45.871706508" watchObservedRunningTime="2026-07-30 09:50:09.395550274 +0000 UTC m=+45.889745758"
Jul 30 09:50:09.395925 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.395700    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78" podStartSLOduration=2.16758648 podStartE2EDuration="9.395694235s" totalImagesPullingTime="7.228107755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:09.394200052 +0000 UTC m=+45.888395559" watchObservedRunningTime="2026-07-30 09:50:09.395694235 +0000 UTC m=+45.889889741"
Jul 30 09:50:09.418196 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.418114    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r" podStartSLOduration=2.549167298 podStartE2EDuration="9.418092626s" totalImagesPullingTime="6.868925328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:09.41751612 +0000 UTC m=+45.911711623" watchObservedRunningTime="2026-07-30 09:50:09.418092626 +0000 UTC m=+45.912288134"
Jul 30 09:50:09.455357 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.455280    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zhg8c" podStartSLOduration=4.236409615 podStartE2EDuration="12.455256081s" totalImagesPullingTime="8.218846466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:49:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:09.454535649 +0000 UTC m=+45.948731207" watchObservedRunningTime="2026-07-30 09:50:09.455256081 +0000 UTC m=+45.949451588"
Jul 30 09:50:09.455631 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.455592    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-tbk9v" podStartSLOduration=2.239563839 podStartE2EDuration="9.45557897s" totalImagesPullingTime="7.216015131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:09.437212307 +0000 UTC m=+45.931407815" watchObservedRunningTime="2026-07-30 09:50:09.45557897 +0000 UTC m=+45.949774477"
Jul 30 09:50:09.677547 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.677466    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cr8nk"]
Jul 30 09:50:09.699230 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.699164    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cr8nk"]
Jul 30 09:50:09.699432 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.699284    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:09.702835 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.702798    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 09:50:09.702955 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.702877    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vvm58\""
Jul 30 09:50:09.703139 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.703123    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 09:50:09.756932 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.756896    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7675\" (UniqueName: \"kubernetes.io/projected/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-api-access-h7675\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:09.757123 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.756958    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:09.757123 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.757060    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:09.757123 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.757100    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca650cbb-da82-43ee-b3fe-397ab13a2a25-data-volume\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:09.757242 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.757129    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ca650cbb-da82-43ee-b3fe-397ab13a2a25-crio-socket\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:09.858467 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.858431    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ca650cbb-da82-43ee-b3fe-397ab13a2a25-crio-socket\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:09.858626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.858564    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7675\" (UniqueName: \"kubernetes.io/projected/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-api-access-h7675\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:09.858626 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.858620    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:09.858723 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.858705    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ca650cbb-da82-43ee-b3fe-397ab13a2a25-crio-socket\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:09.858800 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:09.858783    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 09:50:09.858858 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:09.858850    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm podName:ca650cbb-da82-43ee-b3fe-397ab13a2a25 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:10.358831749 +0000 UTC m=+46.853027240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cr8nk" (UID: "ca650cbb-da82-43ee-b3fe-397ab13a2a25") : configmap "kube-rbac-proxy" not found
Jul 30 09:50:09.858923 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.858894    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:09.858978 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.858937    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca650cbb-da82-43ee-b3fe-397ab13a2a25-data-volume\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:09.859099 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:09.859076    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 09:50:09.859298 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:09.859284    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls podName:ca650cbb-da82-43ee-b3fe-397ab13a2a25 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:10.359265441 +0000 UTC m=+46.853460940 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cr8nk" (UID: "ca650cbb-da82-43ee-b3fe-397ab13a2a25") : secret "insights-runtime-extractor-tls" not found
Jul 30 09:50:09.859425 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.859305    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca650cbb-da82-43ee-b3fe-397ab13a2a25-data-volume\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:09.883328 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:09.883296    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7675\" (UniqueName: \"kubernetes.io/projected/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-api-access-h7675\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:10.017089 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.017014    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-pmclw"]
Jul 30 09:50:10.031660 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.031632    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-pmclw"
Jul 30 09:50:10.034995 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.034971    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 30 09:50:10.035364 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.035020    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 30 09:50:10.035781 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.035763    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-w44n5\""
Jul 30 09:50:10.036900 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.036880    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-pmclw"]
Jul 30 09:50:10.060107 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.060081    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr5tm\" (UniqueName: \"kubernetes.io/projected/f06a6ff0-0ed2-4918-b283-081db15b9418-kube-api-access-vr5tm\") pod \"migrator-6865c9967-pmclw\" (UID: \"f06a6ff0-0ed2-4918-b283-081db15b9418\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-pmclw"
Jul 30 09:50:10.160931 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.160889    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr5tm\" (UniqueName: \"kubernetes.io/projected/f06a6ff0-0ed2-4918-b283-081db15b9418-kube-api-access-vr5tm\") pod \"migrator-6865c9967-pmclw\" (UID: \"f06a6ff0-0ed2-4918-b283-081db15b9418\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-pmclw"
Jul 30 09:50:10.170052 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.170021    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr5tm\" (UniqueName: \"kubernetes.io/projected/f06a6ff0-0ed2-4918-b283-081db15b9418-kube-api-access-vr5tm\") pod \"migrator-6865c9967-pmclw\" (UID: \"f06a6ff0-0ed2-4918-b283-081db15b9418\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-pmclw"
Jul 30 09:50:10.331520 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.331439    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/1.log"
Jul 30 09:50:10.331877 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.331845    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/0.log"
Jul 30 09:50:10.331938 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.331877    2567 generic.go:356] "Generic (PLEG): container finished" podID="97e78a90-a448-4e2d-bdf4-5b21892bb617" containerID="dade5219ef1b26ef6dfd90fcc2b86e780d6f15f4b0b8b67cf2ebcd552b934732" exitCode=255
Jul 30 09:50:10.332011 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.331985    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c" event={"ID":"97e78a90-a448-4e2d-bdf4-5b21892bb617","Type":"ContainerDied","Data":"dade5219ef1b26ef6dfd90fcc2b86e780d6f15f4b0b8b67cf2ebcd552b934732"}
Jul 30 09:50:10.332066 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.332048    2567 scope.go:117] "RemoveContainer" containerID="d96583199d671da41957940613087e6af9221cbee223c29b732c4ddf43afddc7"
Jul 30 09:50:10.332268 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.332244    2567 scope.go:117] "RemoveContainer" containerID="dade5219ef1b26ef6dfd90fcc2b86e780d6f15f4b0b8b67cf2ebcd552b934732"
Jul 30 09:50:10.332487 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:10.332467    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-hw45c_openshift-console-operator(97e78a90-a448-4e2d-bdf4-5b21892bb617)\"" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c" podUID="97e78a90-a448-4e2d-bdf4-5b21892bb617"
Jul 30 09:50:10.341934 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.341915    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-pmclw"
Jul 30 09:50:10.363656 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.363620    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:10.363913 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:10.363704    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 09:50:10.364040 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:10.363952    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm podName:ca650cbb-da82-43ee-b3fe-397ab13a2a25 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:11.363932762 +0000 UTC m=+47.858128260 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cr8nk" (UID: "ca650cbb-da82-43ee-b3fe-397ab13a2a25") : configmap "kube-rbac-proxy" not found
Jul 30 09:50:10.364139 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.364052    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:10.364201 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:10.364137    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 09:50:10.364201 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:10.364198    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls podName:ca650cbb-da82-43ee-b3fe-397ab13a2a25 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:11.364179933 +0000 UTC m=+47.858375418 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cr8nk" (UID: "ca650cbb-da82-43ee-b3fe-397ab13a2a25") : secret "insights-runtime-extractor-tls" not found
Jul 30 09:50:10.497717 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.497641    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-pmclw"]
Jul 30 09:50:10.500265 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:10.500237    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf06a6ff0_0ed2_4918_b283_081db15b9418.slice/crio-be081df623dc68b71a4292cb177efc1eaf2972cc36e51272db881a0c5d1c4b3d WatchSource:0}: Error finding container be081df623dc68b71a4292cb177efc1eaf2972cc36e51272db881a0c5d1c4b3d: Status 404 returned error can't find the container with id be081df623dc68b71a4292cb177efc1eaf2972cc36e51272db881a0c5d1c4b3d
Jul 30 09:50:10.819270 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.819234    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:10.819464 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.819292    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:10.876288 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.876260    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gqhzm_2cfa238d-618e-4dc2-9128-77df904614c9/dns-node-resolver/0.log"
Jul 30 09:50:10.970414 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:10.970379    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fhkxr\" (UID: \"8d349f02-e060-4dca-b9b9-44163a0fb935\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"
Jul 30 09:50:10.970559 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:10.970527    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 09:50:10.970616 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:10.970600    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert podName:8d349f02-e060-4dca-b9b9-44163a0fb935 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:18.970584709 +0000 UTC m=+55.464780193 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-fhkxr" (UID: "8d349f02-e060-4dca-b9b9-44163a0fb935") : secret "networking-console-plugin-cert" not found
Jul 30 09:50:11.337814 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:11.337787    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/1.log"
Jul 30 09:50:11.338285 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:11.338141    2567 scope.go:117] "RemoveContainer" containerID="dade5219ef1b26ef6dfd90fcc2b86e780d6f15f4b0b8b67cf2ebcd552b934732"
Jul 30 09:50:11.338404 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:11.338325    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-hw45c_openshift-console-operator(97e78a90-a448-4e2d-bdf4-5b21892bb617)\"" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c" podUID="97e78a90-a448-4e2d-bdf4-5b21892bb617"
Jul 30 09:50:11.339104 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:11.339085    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-pmclw" event={"ID":"f06a6ff0-0ed2-4918-b283-081db15b9418","Type":"ContainerStarted","Data":"be081df623dc68b71a4292cb177efc1eaf2972cc36e51272db881a0c5d1c4b3d"}
Jul 30 09:50:11.373478 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:11.373429    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:11.373644 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:11.373526    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 09:50:11.373644 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:11.373572    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:11.373644 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:11.373601    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm podName:ca650cbb-da82-43ee-b3fe-397ab13a2a25 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:13.373579183 +0000 UTC m=+49.867774670 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cr8nk" (UID: "ca650cbb-da82-43ee-b3fe-397ab13a2a25") : configmap "kube-rbac-proxy" not found
Jul 30 09:50:11.373854 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:11.373786    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 09:50:11.373854 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:11.373834    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls podName:ca650cbb-da82-43ee-b3fe-397ab13a2a25 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:13.373820237 +0000 UTC m=+49.868015728 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cr8nk" (UID: "ca650cbb-da82-43ee-b3fe-397ab13a2a25") : secret "insights-runtime-extractor-tls" not found
Jul 30 09:50:11.876647 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:11.876616    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lt2xc_f556c8a9-3801-4f40-b5d7-1ffa2701eed1/node-ca/0.log"
Jul 30 09:50:12.216374 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.216323    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-k86ms"]
Jul 30 09:50:12.235637 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.235609    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-k86ms"
Jul 30 09:50:12.236735 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.236708    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-k86ms"]
Jul 30 09:50:12.238410 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.238387    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 30 09:50:12.238512 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.238387    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 30 09:50:12.239437 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.239406    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-nxpbc\""
Jul 30 09:50:12.239547 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.239456    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 30 09:50:12.239547 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.239456    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 30 09:50:12.283501 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.283469    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvwzr\" (UniqueName: \"kubernetes.io/projected/8ad33777-81e7-4340-8e68-d124c41bf143-kube-api-access-cvwzr\") pod \"service-ca-658d7b7c78-k86ms\" (UID: \"8ad33777-81e7-4340-8e68-d124c41bf143\") " pod="openshift-service-ca/service-ca-658d7b7c78-k86ms"
Jul 30 09:50:12.283667 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.283509    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8ad33777-81e7-4340-8e68-d124c41bf143-signing-cabundle\") pod \"service-ca-658d7b7c78-k86ms\" (UID: \"8ad33777-81e7-4340-8e68-d124c41bf143\") " pod="openshift-service-ca/service-ca-658d7b7c78-k86ms"
Jul 30 09:50:12.283667 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.283614    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8ad33777-81e7-4340-8e68-d124c41bf143-signing-key\") pod \"service-ca-658d7b7c78-k86ms\" (UID: \"8ad33777-81e7-4340-8e68-d124c41bf143\") " pod="openshift-service-ca/service-ca-658d7b7c78-k86ms"
Jul 30 09:50:12.342992 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.342963    2567 scope.go:117] "RemoveContainer" containerID="dade5219ef1b26ef6dfd90fcc2b86e780d6f15f4b0b8b67cf2ebcd552b934732"
Jul 30 09:50:12.343376 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:12.343131    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-hw45c_openshift-console-operator(97e78a90-a448-4e2d-bdf4-5b21892bb617)\"" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c" podUID="97e78a90-a448-4e2d-bdf4-5b21892bb617"
Jul 30 09:50:12.384560 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.384523    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvwzr\" (UniqueName: \"kubernetes.io/projected/8ad33777-81e7-4340-8e68-d124c41bf143-kube-api-access-cvwzr\") pod \"service-ca-658d7b7c78-k86ms\" (UID: \"8ad33777-81e7-4340-8e68-d124c41bf143\") " pod="openshift-service-ca/service-ca-658d7b7c78-k86ms"
Jul 30 09:50:12.384560 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.384564    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8ad33777-81e7-4340-8e68-d124c41bf143-signing-cabundle\") pod \"service-ca-658d7b7c78-k86ms\" (UID: \"8ad33777-81e7-4340-8e68-d124c41bf143\") " pod="openshift-service-ca/service-ca-658d7b7c78-k86ms"
Jul 30 09:50:12.384931 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.384897    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8ad33777-81e7-4340-8e68-d124c41bf143-signing-key\") pod \"service-ca-658d7b7c78-k86ms\" (UID: \"8ad33777-81e7-4340-8e68-d124c41bf143\") " pod="openshift-service-ca/service-ca-658d7b7c78-k86ms"
Jul 30 09:50:12.385226 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.385204    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8ad33777-81e7-4340-8e68-d124c41bf143-signing-cabundle\") pod \"service-ca-658d7b7c78-k86ms\" (UID: \"8ad33777-81e7-4340-8e68-d124c41bf143\") " pod="openshift-service-ca/service-ca-658d7b7c78-k86ms"
Jul 30 09:50:12.387328 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.387305    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8ad33777-81e7-4340-8e68-d124c41bf143-signing-key\") pod \"service-ca-658d7b7c78-k86ms\" (UID: \"8ad33777-81e7-4340-8e68-d124c41bf143\") " pod="openshift-service-ca/service-ca-658d7b7c78-k86ms"
Jul 30 09:50:12.393643 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.393616    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvwzr\" (UniqueName: \"kubernetes.io/projected/8ad33777-81e7-4340-8e68-d124c41bf143-kube-api-access-cvwzr\") pod \"service-ca-658d7b7c78-k86ms\" (UID: \"8ad33777-81e7-4340-8e68-d124c41bf143\") " pod="openshift-service-ca/service-ca-658d7b7c78-k86ms"
Jul 30 09:50:12.477014 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.476927    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zhg8c_c89418e7-06ea-4bb2-8c58-22fbd57034b5/serve-healthcheck-canary/0.log"
Jul 30 09:50:12.546115 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.546089    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-k86ms"
Jul 30 09:50:12.701731 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:12.701671    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-k86ms"]
Jul 30 09:50:12.704014 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:12.703984    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad33777_81e7_4340_8e68_d124c41bf143.slice/crio-603fe52d8a0d0704ef09b087007309212d69660000c60a88804ad16873e7b270 WatchSource:0}: Error finding container 603fe52d8a0d0704ef09b087007309212d69660000c60a88804ad16873e7b270: Status 404 returned error can't find the container with id 603fe52d8a0d0704ef09b087007309212d69660000c60a88804ad16873e7b270
Jul 30 09:50:13.348297 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:13.348262    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-pmclw" event={"ID":"f06a6ff0-0ed2-4918-b283-081db15b9418","Type":"ContainerStarted","Data":"1db991fb210afe2d9352372c837e888877a98bc506c111200c74d189a8b446ec"}
Jul 30 09:50:13.348753 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:13.348321    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-pmclw" event={"ID":"f06a6ff0-0ed2-4918-b283-081db15b9418","Type":"ContainerStarted","Data":"db12519fd6b309ca634e644f6273a2075bcb1cb209e52b36f8dbb240d5141470"}
Jul 30 09:50:13.349820 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:13.349794    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-k86ms" event={"ID":"8ad33777-81e7-4340-8e68-d124c41bf143","Type":"ContainerStarted","Data":"b9c67d6be41c2d4649a8c59182fd448288a4d2564a49eac410bb6fff5de5cc4d"}
Jul 30 09:50:13.349948 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:13.349839    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-k86ms" event={"ID":"8ad33777-81e7-4340-8e68-d124c41bf143","Type":"ContainerStarted","Data":"603fe52d8a0d0704ef09b087007309212d69660000c60a88804ad16873e7b270"}
Jul 30 09:50:13.364379 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:13.364323    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-pmclw" podStartSLOduration=2.359936054 podStartE2EDuration="4.36430747s" totalImagesPullingTime="2.004371416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:13.362772349 +0000 UTC m=+49.856967856" watchObservedRunningTime="2026-07-30 09:50:13.36430747 +0000 UTC m=+49.858502976"
Jul 30 09:50:13.377753 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:13.377705    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-k86ms" podStartSLOduration=1.377688043 podStartE2EDuration="1.377688043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:13.377000409 +0000 UTC m=+49.871195937" watchObservedRunningTime="2026-07-30 09:50:13.377688043 +0000 UTC m=+49.871883548"
Jul 30 09:50:13.394792 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:13.394754    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:13.394969 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:13.394871    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:13.395047 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:13.394961    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 30 09:50:13.395105 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:13.395045    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm podName:ca650cbb-da82-43ee-b3fe-397ab13a2a25 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:17.39502312 +0000 UTC m=+53.889218618 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cr8nk" (UID: "ca650cbb-da82-43ee-b3fe-397ab13a2a25") : configmap "kube-rbac-proxy" not found
Jul 30 09:50:13.395171 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:13.395144    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 09:50:13.395225 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:13.395196    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls podName:ca650cbb-da82-43ee-b3fe-397ab13a2a25 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:17.395182036 +0000 UTC m=+53.889377539 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cr8nk" (UID: "ca650cbb-da82-43ee-b3fe-397ab13a2a25") : secret "insights-runtime-extractor-tls" not found
Jul 30 09:50:13.698265 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:13.698217    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:50:13.698508 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:13.698441    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 30 09:50:13.698508 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:13.698456    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-598bfb974b-kzqfn: secret "image-registry-tls" not found
Jul 30 09:50:13.698508 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:13.698501    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls podName:2f923caa-9667-4c0d-8083-47944d143e9d nodeName:}" failed. No retries permitted until 2026-07-30 09:50:29.698486074 +0000 UTC m=+66.192681558 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls") pod "image-registry-598bfb974b-kzqfn" (UID: "2f923caa-9667-4c0d-8083-47944d143e9d") : secret "image-registry-tls" not found
Jul 30 09:50:13.799676 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:13.799633    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:50:13.800125 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:13.800004    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 30 09:50:13.800733 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:13.800709    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls podName:3c1a1aad-170c-4e52-8f7d-e763663b81e4 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:29.800684563 +0000 UTC m=+66.294880062 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls") pod "dns-default-84b5h" (UID: "3c1a1aad-170c-4e52-8f7d-e763663b81e4") : secret "dns-default-metrics-tls" not found
Jul 30 09:50:16.224722 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:16.224675    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bwpth\" (UID: \"b6d08841-c5eb-4c9e-a1cd-ffaaae91d021\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"
Jul 30 09:50:16.237235 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:16.237210    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6d08841-c5eb-4c9e-a1cd-ffaaae91d021-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-bwpth\" (UID: \"b6d08841-c5eb-4c9e-a1cd-ffaaae91d021\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"
Jul 30 09:50:16.326116 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:16.326075    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:16.326322 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:16.326227    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 30 09:50:16.326322 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:16.326305    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls podName:da4c528d-7534-4727-8ace-474996ab3454 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:32.326288479 +0000 UTC m=+68.820483967 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-mz9gf" (UID: "da4c528d-7534-4727-8ace-474996ab3454") : secret "cluster-monitoring-operator-tls" not found
Jul 30 09:50:16.427471 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:16.427424    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:16.427665 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:16.427487    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:16.427665 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:16.427573    2567 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 30 09:50:16.427665 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:16.427608    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle podName:b19ed290-2519-455d-8147-688ac9ed11b0 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:32.427592444 +0000 UTC m=+68.921787932 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle") pod "router-default-888465b95-k24rp" (UID: "b19ed290-2519-455d-8147-688ac9ed11b0") : configmap references non-existent config key: service-ca.crt
Jul 30 09:50:16.427665 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:16.427627    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs podName:b19ed290-2519-455d-8147-688ac9ed11b0 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:32.42762031 +0000 UTC m=+68.921815794 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs") pod "router-default-888465b95-k24rp" (UID: "b19ed290-2519-455d-8147-688ac9ed11b0") : secret "router-metrics-certs-default" not found
Jul 30 09:50:16.517790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:16.517696    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"
Jul 30 09:50:16.672258 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:16.672231    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth"]
Jul 30 09:50:17.366649 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:17.366613    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth" event={"ID":"b6d08841-c5eb-4c9e-a1cd-ffaaae91d021","Type":"ContainerStarted","Data":"9591833d4607c6e9880d8c1572686c20f9951496f2aa38802f356e942355dc34"}
Jul 30 09:50:17.435557 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:17.435518    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:17.435744 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:17.435605    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:17.435744 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:17.435710    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 30 09:50:17.435954 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:17.435929    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls podName:ca650cbb-da82-43ee-b3fe-397ab13a2a25 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:25.435898299 +0000 UTC m=+61.930093792 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cr8nk" (UID: "ca650cbb-da82-43ee-b3fe-397ab13a2a25") : secret "insights-runtime-extractor-tls" not found
Jul 30 09:50:17.436414 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:17.436123    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ca650cbb-da82-43ee-b3fe-397ab13a2a25-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:19.053716 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:19.053677    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fhkxr\" (UID: \"8d349f02-e060-4dca-b9b9-44163a0fb935\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"
Jul 30 09:50:19.054119 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:19.053843    2567 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 30 09:50:19.054119 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:19.053909    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert podName:8d349f02-e060-4dca-b9b9-44163a0fb935 nodeName:}" failed. No retries permitted until 2026-07-30 09:50:35.053892642 +0000 UTC m=+71.548088147 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-fhkxr" (UID: "8d349f02-e060-4dca-b9b9-44163a0fb935") : secret "networking-console-plugin-cert" not found
Jul 30 09:50:19.374935 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:19.374894    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth" event={"ID":"b6d08841-c5eb-4c9e-a1cd-ffaaae91d021","Type":"ContainerStarted","Data":"f4b6ced841b19d0a600ab744f6a962ae7a25e6a81a543c568b95a4ddcfbb7227"}
Jul 30 09:50:19.375225 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:19.374973    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth" event={"ID":"b6d08841-c5eb-4c9e-a1cd-ffaaae91d021","Type":"ContainerStarted","Data":"f31538c8b85b501aebb65d2cefe75037c2177ee6ebb54d180717b22f768ff716"}
Jul 30 09:50:19.393002 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:19.392940    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-bwpth" podStartSLOduration=17.579103937 podStartE2EDuration="19.392921937s" totalImagesPullingTime="1.813818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:19.391826962 +0000 UTC m=+55.886022470" watchObservedRunningTime="2026-07-30 09:50:19.392921937 +0000 UTC m=+55.887117471"
Jul 30 09:50:24.263212 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:24.263177    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wf4hv"
Jul 30 09:50:25.057731 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:25.057697    2567 scope.go:117] "RemoveContainer" containerID="dade5219ef1b26ef6dfd90fcc2b86e780d6f15f4b0b8b67cf2ebcd552b934732"
Jul 30 09:50:25.397963 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:25.397932    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 09:50:25.398489 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:25.398472    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/1.log"
Jul 30 09:50:25.398558 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:25.398508    2567 generic.go:356] "Generic (PLEG): container finished" podID="97e78a90-a448-4e2d-bdf4-5b21892bb617" containerID="91116bcc022445da3513f72e9b41661ca25476bbe99307fad052abee0fca233e" exitCode=255
Jul 30 09:50:25.398612 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:25.398553    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c" event={"ID":"97e78a90-a448-4e2d-bdf4-5b21892bb617","Type":"ContainerDied","Data":"91116bcc022445da3513f72e9b41661ca25476bbe99307fad052abee0fca233e"}
Jul 30 09:50:25.398657 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:25.398634    2567 scope.go:117] "RemoveContainer" containerID="dade5219ef1b26ef6dfd90fcc2b86e780d6f15f4b0b8b67cf2ebcd552b934732"
Jul 30 09:50:25.398990 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:25.398973    2567 scope.go:117] "RemoveContainer" containerID="91116bcc022445da3513f72e9b41661ca25476bbe99307fad052abee0fca233e"
Jul 30 09:50:25.399237 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:25.399197    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-hw45c_openshift-console-operator(97e78a90-a448-4e2d-bdf4-5b21892bb617)\"" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c" podUID="97e78a90-a448-4e2d-bdf4-5b21892bb617"
Jul 30 09:50:25.513065 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:25.513020    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:25.515283 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:25.515261    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ca650cbb-da82-43ee-b3fe-397ab13a2a25-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cr8nk\" (UID: \"ca650cbb-da82-43ee-b3fe-397ab13a2a25\") " pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:25.613813 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:25.613775    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vvm58\""
Jul 30 09:50:25.621925 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:25.621893    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cr8nk"
Jul 30 09:50:25.748296 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:25.748265    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cr8nk"]
Jul 30 09:50:25.753669 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:25.753639    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca650cbb_da82_43ee_b3fe_397ab13a2a25.slice/crio-8fa0fbcd69769d9a2eab2f72683e75157d2117d7a35e47ec167b134479e9f551 WatchSource:0}: Error finding container 8fa0fbcd69769d9a2eab2f72683e75157d2117d7a35e47ec167b134479e9f551: Status 404 returned error can't find the container with id 8fa0fbcd69769d9a2eab2f72683e75157d2117d7a35e47ec167b134479e9f551
Jul 30 09:50:26.405331 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:26.405294    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 09:50:26.408221 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:26.407914    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cr8nk" event={"ID":"ca650cbb-da82-43ee-b3fe-397ab13a2a25","Type":"ContainerStarted","Data":"8fa0fbcd69769d9a2eab2f72683e75157d2117d7a35e47ec167b134479e9f551"}
Jul 30 09:50:28.640749 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:28.640707    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:50:28.644966 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:28.644940    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 09:50:28.653329 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:28.653303    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/807d7a19-7b79-4963-a013-53d46d57b8a2-original-pull-secret\") pod \"global-pull-secret-syncer-dfptt\" (UID: \"807d7a19-7b79-4963-a013-53d46d57b8a2\") " pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:50:28.668508 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:28.668473    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfptt"
Jul 30 09:50:28.742091 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:28.742049    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs\") pod \"network-metrics-daemon-bf2q6\" (UID: \"02c86fe5-e1d2-486d-adbc-1ed595aa0f72\") " pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:50:28.745069 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:28.745042    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 09:50:28.755775 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:28.755745    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02c86fe5-e1d2-486d-adbc-1ed595aa0f72-metrics-certs\") pod \"network-metrics-daemon-bf2q6\" (UID: \"02c86fe5-e1d2-486d-adbc-1ed595aa0f72\") " pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:50:28.776723 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:28.776692    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fxw5q\""
Jul 30 09:50:28.785029 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:28.784999    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bf2q6"
Jul 30 09:50:28.843114 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:28.843067    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z2cn\" (UniqueName: \"kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn\") pod \"network-check-target-gxth2\" (UID: \"8ade6f03-fb02-44bd-864e-7bdfc5df23b7\") " pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:50:28.846013 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:28.845977    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z2cn\" (UniqueName: \"kubernetes.io/projected/8ade6f03-fb02-44bd-864e-7bdfc5df23b7-kube-api-access-6z2cn\") pod \"network-check-target-gxth2\" (UID: \"8ade6f03-fb02-44bd-864e-7bdfc5df23b7\") " pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:50:29.072534 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:29.072452    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l5pnh\""
Jul 30 09:50:29.079088 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:29.079056    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:50:29.753033 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:29.752999    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:50:29.755459 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:29.755436    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls\") pod \"image-registry-598bfb974b-kzqfn\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") " pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:50:29.853974 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:29.853939    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:50:29.856104 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:29.856083    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c1a1aad-170c-4e52-8f7d-e763663b81e4-metrics-tls\") pod \"dns-default-84b5h\" (UID: \"3c1a1aad-170c-4e52-8f7d-e763663b81e4\") " pod="openshift-dns/dns-default-84b5h"
Jul 30 09:50:29.957964 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:29.957935    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6fk8k\""
Jul 30 09:50:29.965037 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:29.965009    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:50:29.982060 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:29.982030    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cw2kw\""
Jul 30 09:50:29.990113 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:29.990087    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-84b5h"
Jul 30 09:50:30.794176 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:30.794141    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gxth2"]
Jul 30 09:50:30.818772 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:30.818725    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:30.818967 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:30.818816    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:30.819214 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:30.819196    2567 scope.go:117] "RemoveContainer" containerID="91116bcc022445da3513f72e9b41661ca25476bbe99307fad052abee0fca233e"
Jul 30 09:50:30.819409 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:50:30.819389    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-hw45c_openshift-console-operator(97e78a90-a448-4e2d-bdf4-5b21892bb617)\"" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c" podUID="97e78a90-a448-4e2d-bdf4-5b21892bb617"
Jul 30 09:50:30.943848 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:30.943818    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ade6f03_fb02_44bd_864e_7bdfc5df23b7.slice/crio-fba8f020a377e4bf99e7b4ce49950d15e77a02937b5d2ba58624952bac56826f WatchSource:0}: Error finding container fba8f020a377e4bf99e7b4ce49950d15e77a02937b5d2ba58624952bac56826f: Status 404 returned error can't find the container with id fba8f020a377e4bf99e7b4ce49950d15e77a02937b5d2ba58624952bac56826f
Jul 30 09:50:31.031007 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.030980    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-84b5h"]
Jul 30 09:50:31.031693 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:31.031667    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c1a1aad_170c_4e52_8f7d_e763663b81e4.slice/crio-b6c78f27e6913e4424b25f480d7eef5c70870e268be45a633e67668c0c757046 WatchSource:0}: Error finding container b6c78f27e6913e4424b25f480d7eef5c70870e268be45a633e67668c0c757046: Status 404 returned error can't find the container with id b6c78f27e6913e4424b25f480d7eef5c70870e268be45a633e67668c0c757046
Jul 30 09:50:31.033921 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.033783    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dfptt"]
Jul 30 09:50:31.036812 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.036714    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-598bfb974b-kzqfn"]
Jul 30 09:50:31.037970 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:31.037941    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod807d7a19_7b79_4963_a013_53d46d57b8a2.slice/crio-9032e74c707bcff78c541d356072149b642c92a7bc018eb435b5671440d4bc15 WatchSource:0}: Error finding container 9032e74c707bcff78c541d356072149b642c92a7bc018eb435b5671440d4bc15: Status 404 returned error can't find the container with id 9032e74c707bcff78c541d356072149b642c92a7bc018eb435b5671440d4bc15
Jul 30 09:50:31.041459 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.039580    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bf2q6"]
Jul 30 09:50:31.050360 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:31.050321    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02c86fe5_e1d2_486d_adbc_1ed595aa0f72.slice/crio-afef9ef57b8e39bd68481e494590e4346e849a4aa5c5bee72d416cfe9608d211 WatchSource:0}: Error finding container afef9ef57b8e39bd68481e494590e4346e849a4aa5c5bee72d416cfe9608d211: Status 404 returned error can't find the container with id afef9ef57b8e39bd68481e494590e4346e849a4aa5c5bee72d416cfe9608d211
Jul 30 09:50:31.430298 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.430260    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gxth2" event={"ID":"8ade6f03-fb02-44bd-864e-7bdfc5df23b7","Type":"ContainerStarted","Data":"fddec5b41f21d3fc309f09f29dd70351d81725ca89e7c8eb25704ed1315bd746"}
Jul 30 09:50:31.430494 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.430324    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gxth2" event={"ID":"8ade6f03-fb02-44bd-864e-7bdfc5df23b7","Type":"ContainerStarted","Data":"fba8f020a377e4bf99e7b4ce49950d15e77a02937b5d2ba58624952bac56826f"}
Jul 30 09:50:31.430494 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.430367    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:50:31.431852 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.431828    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-598bfb974b-kzqfn" event={"ID":"2f923caa-9667-4c0d-8083-47944d143e9d","Type":"ContainerStarted","Data":"cf11c60bb8e7ca7ecd8ffb8251cd8afaf62c891adbbcf372f05ccddec1bd1e5d"}
Jul 30 09:50:31.431978 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.431873    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-598bfb974b-kzqfn" event={"ID":"2f923caa-9667-4c0d-8083-47944d143e9d","Type":"ContainerStarted","Data":"61ea1f1b118c90f88b4577f97d7dd2f85ec7d203361181f95f726f488566b4d8"}
Jul 30 09:50:31.431978 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.431896    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:50:31.432949 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.432927    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-84b5h" event={"ID":"3c1a1aad-170c-4e52-8f7d-e763663b81e4","Type":"ContainerStarted","Data":"b6c78f27e6913e4424b25f480d7eef5c70870e268be45a633e67668c0c757046"}
Jul 30 09:50:31.434113 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.434080    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bf2q6" event={"ID":"02c86fe5-e1d2-486d-adbc-1ed595aa0f72","Type":"ContainerStarted","Data":"afef9ef57b8e39bd68481e494590e4346e849a4aa5c5bee72d416cfe9608d211"}
Jul 30 09:50:31.435560 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.435541    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cr8nk" event={"ID":"ca650cbb-da82-43ee-b3fe-397ab13a2a25","Type":"ContainerStarted","Data":"0d6a0668f3e4367164bce04d90d82a4f3809fdbabe15e91a223211c0b917b020"}
Jul 30 09:50:31.436541 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.436510    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dfptt" event={"ID":"807d7a19-7b79-4963-a013-53d46d57b8a2","Type":"ContainerStarted","Data":"9032e74c707bcff78c541d356072149b642c92a7bc018eb435b5671440d4bc15"}
Jul 30 09:50:31.447905 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.447840    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gxth2" podStartSLOduration=67.447816148 podStartE2EDuration="1m7.447816148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:31.446001988 +0000 UTC m=+67.940197498" watchObservedRunningTime="2026-07-30 09:50:31.447816148 +0000 UTC m=+67.942011653"
Jul 30 09:50:31.466037 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:31.465987    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-598bfb974b-kzqfn" podStartSLOduration=67.465970231 podStartE2EDuration="1m7.465970231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:31.465723404 +0000 UTC m=+67.959918924" watchObservedRunningTime="2026-07-30 09:50:31.465970231 +0000 UTC m=+67.960165739"
Jul 30 09:50:32.318248 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.318210    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n"]
Jul 30 09:50:32.337152 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.337121    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"]
Jul 30 09:50:32.337468 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.337442    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n"
Jul 30 09:50:32.341943 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.341913    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 30 09:50:32.343302 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.342643    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 30 09:50:32.343302 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.342870    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 30 09:50:32.343302 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.343083    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 30 09:50:32.344238 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.343839    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-sqj78\""
Jul 30 09:50:32.357205 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.357177    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"]
Jul 30 09:50:32.357361 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.357223    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"
Jul 30 09:50:32.360231 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.360207    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 30 09:50:32.376538 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.376509    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:32.377637 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.377574    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n"]
Jul 30 09:50:32.377637 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.377609    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"]
Jul 30 09:50:32.377858 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.377710    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.378551 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.378393    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"]
Jul 30 09:50:32.379761 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.379712    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da4c528d-7534-4727-8ace-474996ab3454-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-mz9gf\" (UID: \"da4c528d-7534-4727-8ace-474996ab3454\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:32.380803 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.380781    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 30 09:50:32.381183 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.381018    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 30 09:50:32.381263 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.381211    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 30 09:50:32.381507 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.381487    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 30 09:50:32.477552 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.477518    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e64d61ea-9828-4770-827b-32abb45f45e5-hub\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.477729 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.477569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/31a81bf8-7bdf-4056-8a66-2de189f419a9-klusterlet-config\") pod \"klusterlet-addon-workmgr-764f8fcd77-r5f7b\" (UID: \"31a81bf8-7bdf-4056-8a66-2de189f419a9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"
Jul 30 09:50:32.477729 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.477661    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e64d61ea-9828-4770-827b-32abb45f45e5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.477850 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.477731    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cjcp\" (UniqueName: \"kubernetes.io/projected/e64d61ea-9828-4770-827b-32abb45f45e5-kube-api-access-5cjcp\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.477850 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.477783    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e64d61ea-9828-4770-827b-32abb45f45e5-ca\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.477850 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.477811    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e64d61ea-9828-4770-827b-32abb45f45e5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.477956 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.477861    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1b2a919-5411-474f-ad20-251fe1d2501a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n\" (UID: \"d1b2a919-5411-474f-ad20-251fe1d2501a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n"
Jul 30 09:50:32.477956 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.477909    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:32.477956 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.477936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfpcr\" (UniqueName: \"kubernetes.io/projected/31a81bf8-7bdf-4056-8a66-2de189f419a9-kube-api-access-dfpcr\") pod \"klusterlet-addon-workmgr-764f8fcd77-r5f7b\" (UID: \"31a81bf8-7bdf-4056-8a66-2de189f419a9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"
Jul 30 09:50:32.478054 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.477984    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:32.478054 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.478031    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e64d61ea-9828-4770-827b-32abb45f45e5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.478137 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.478052    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31a81bf8-7bdf-4056-8a66-2de189f419a9-tmp\") pod \"klusterlet-addon-workmgr-764f8fcd77-r5f7b\" (UID: \"31a81bf8-7bdf-4056-8a66-2de189f419a9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"
Jul 30 09:50:32.478137 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.478100    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v26kc\" (UniqueName: \"kubernetes.io/projected/d1b2a919-5411-474f-ad20-251fe1d2501a-kube-api-access-v26kc\") pod \"managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n\" (UID: \"d1b2a919-5411-474f-ad20-251fe1d2501a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n"
Jul 30 09:50:32.478841 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.478820    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b19ed290-2519-455d-8147-688ac9ed11b0-service-ca-bundle\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:32.482087 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.482027    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19ed290-2519-455d-8147-688ac9ed11b0-metrics-certs\") pod \"router-default-888465b95-k24rp\" (UID: \"b19ed290-2519-455d-8147-688ac9ed11b0\") " pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:32.579844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.579069    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e64d61ea-9828-4770-827b-32abb45f45e5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.579844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.579112    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31a81bf8-7bdf-4056-8a66-2de189f419a9-tmp\") pod \"klusterlet-addon-workmgr-764f8fcd77-r5f7b\" (UID: \"31a81bf8-7bdf-4056-8a66-2de189f419a9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"
Jul 30 09:50:32.579844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.579182    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v26kc\" (UniqueName: \"kubernetes.io/projected/d1b2a919-5411-474f-ad20-251fe1d2501a-kube-api-access-v26kc\") pod \"managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n\" (UID: \"d1b2a919-5411-474f-ad20-251fe1d2501a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n"
Jul 30 09:50:32.579844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.579259    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e64d61ea-9828-4770-827b-32abb45f45e5-hub\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.579844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.579306    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/31a81bf8-7bdf-4056-8a66-2de189f419a9-klusterlet-config\") pod \"klusterlet-addon-workmgr-764f8fcd77-r5f7b\" (UID: \"31a81bf8-7bdf-4056-8a66-2de189f419a9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"
Jul 30 09:50:32.579844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.579406    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e64d61ea-9828-4770-827b-32abb45f45e5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.579844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.579437    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cjcp\" (UniqueName: \"kubernetes.io/projected/e64d61ea-9828-4770-827b-32abb45f45e5-kube-api-access-5cjcp\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.579844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.579476    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e64d61ea-9828-4770-827b-32abb45f45e5-ca\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.579844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.579504    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e64d61ea-9828-4770-827b-32abb45f45e5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.579844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.579541    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1b2a919-5411-474f-ad20-251fe1d2501a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n\" (UID: \"d1b2a919-5411-474f-ad20-251fe1d2501a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n"
Jul 30 09:50:32.579844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.579569    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfpcr\" (UniqueName: \"kubernetes.io/projected/31a81bf8-7bdf-4056-8a66-2de189f419a9-kube-api-access-dfpcr\") pod \"klusterlet-addon-workmgr-764f8fcd77-r5f7b\" (UID: \"31a81bf8-7bdf-4056-8a66-2de189f419a9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"
Jul 30 09:50:32.579844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.579835    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31a81bf8-7bdf-4056-8a66-2de189f419a9-tmp\") pod \"klusterlet-addon-workmgr-764f8fcd77-r5f7b\" (UID: \"31a81bf8-7bdf-4056-8a66-2de189f419a9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"
Jul 30 09:50:32.579844 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.579849    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e64d61ea-9828-4770-827b-32abb45f45e5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.582737 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.582696    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e64d61ea-9828-4770-827b-32abb45f45e5-ca\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.583152 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.583126    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e64d61ea-9828-4770-827b-32abb45f45e5-hub\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.584546 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.584501    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1b2a919-5411-474f-ad20-251fe1d2501a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n\" (UID: \"d1b2a919-5411-474f-ad20-251fe1d2501a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n"
Jul 30 09:50:32.585016 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.584765    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e64d61ea-9828-4770-827b-32abb45f45e5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.585610 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.585578    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e64d61ea-9828-4770-827b-32abb45f45e5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.585714 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.585611    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/31a81bf8-7bdf-4056-8a66-2de189f419a9-klusterlet-config\") pod \"klusterlet-addon-workmgr-764f8fcd77-r5f7b\" (UID: \"31a81bf8-7bdf-4056-8a66-2de189f419a9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"
Jul 30 09:50:32.590841 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.590511    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ppvvv\""
Jul 30 09:50:32.593543 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.593517    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cjcp\" (UniqueName: \"kubernetes.io/projected/e64d61ea-9828-4770-827b-32abb45f45e5-kube-api-access-5cjcp\") pod \"cluster-proxy-proxy-agent-78fd754fbb-5q988\" (UID: \"e64d61ea-9828-4770-827b-32abb45f45e5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.594976 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.594927    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v26kc\" (UniqueName: \"kubernetes.io/projected/d1b2a919-5411-474f-ad20-251fe1d2501a-kube-api-access-v26kc\") pod \"managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n\" (UID: \"d1b2a919-5411-474f-ad20-251fe1d2501a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n"
Jul 30 09:50:32.596314 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.596289    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfpcr\" (UniqueName: \"kubernetes.io/projected/31a81bf8-7bdf-4056-8a66-2de189f419a9-kube-api-access-dfpcr\") pod \"klusterlet-addon-workmgr-764f8fcd77-r5f7b\" (UID: \"31a81bf8-7bdf-4056-8a66-2de189f419a9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"
Jul 30 09:50:32.599188 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.598995    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"
Jul 30 09:50:32.637019 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.636750    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qzgp8\""
Jul 30 09:50:32.645253 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.644953    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:32.661980 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.661928    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n"
Jul 30 09:50:32.673192 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.673104    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"
Jul 30 09:50:32.702979 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.700290    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"
Jul 30 09:50:32.856727 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.843565    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf"]
Jul 30 09:50:32.941252 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.937358    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-888465b95-k24rp"]
Jul 30 09:50:32.964446 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.964392    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n"]
Jul 30 09:50:32.987622 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:32.987600    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"]
Jul 30 09:50:33.012596 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:33.012533    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988"]
Jul 30 09:50:33.018326 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:33.018296    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode64d61ea_9828_4770_827b_32abb45f45e5.slice/crio-0cd3ca3501505769865830ac3797c93ed11bbb2cbae35693d6796941ca640147 WatchSource:0}: Error finding container 0cd3ca3501505769865830ac3797c93ed11bbb2cbae35693d6796941ca640147: Status 404 returned error can't find the container with id 0cd3ca3501505769865830ac3797c93ed11bbb2cbae35693d6796941ca640147
Jul 30 09:50:33.448259 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:33.448218    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n" event={"ID":"d1b2a919-5411-474f-ad20-251fe1d2501a","Type":"ContainerStarted","Data":"00b0c63a702871a910b98453956c2ff191e49a191bcff7e946a0b88a21b4fd44"}
Jul 30 09:50:33.450984 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:33.450939    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bf2q6" event={"ID":"02c86fe5-e1d2-486d-adbc-1ed595aa0f72","Type":"ContainerStarted","Data":"85fba5eb8dfb2a7b588c49e56ba57fd5786cbd42803d7beb898de9144b935193"}
Jul 30 09:50:33.451120 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:33.450997    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bf2q6" event={"ID":"02c86fe5-e1d2-486d-adbc-1ed595aa0f72","Type":"ContainerStarted","Data":"085e55db1e62fec64a5d85791b455bd002a1e5302593496642904100d397b0fc"}
Jul 30 09:50:33.454152 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:33.454090    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988" event={"ID":"e64d61ea-9828-4770-827b-32abb45f45e5","Type":"ContainerStarted","Data":"0cd3ca3501505769865830ac3797c93ed11bbb2cbae35693d6796941ca640147"}
Jul 30 09:50:33.456831 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:33.456798    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b" event={"ID":"31a81bf8-7bdf-4056-8a66-2de189f419a9","Type":"ContainerStarted","Data":"75b04bf385eadfb3abf5e91f9e0ebab77164e7a14cccb90d7585521447194623"}
Jul 30 09:50:33.459569 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:33.459502    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-888465b95-k24rp" event={"ID":"b19ed290-2519-455d-8147-688ac9ed11b0","Type":"ContainerStarted","Data":"a7d30da4a19d8165a05d8cfef118f0e2a63a73cb450c74f7c998057fc3e01f92"}
Jul 30 09:50:33.459569 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:33.459548    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-888465b95-k24rp" event={"ID":"b19ed290-2519-455d-8147-688ac9ed11b0","Type":"ContainerStarted","Data":"2cde2670c3292facad05c91471522f7788efaf4812efcfbfad6ee5fa6d44734e"}
Jul 30 09:50:33.461589 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:33.461534    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf" event={"ID":"da4c528d-7534-4727-8ace-474996ab3454","Type":"ContainerStarted","Data":"183d4a4e9150104bbdeebbbb680988112a78d84895617cac9231d8c2abf57b5c"}
Jul 30 09:50:33.473267 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:33.473058    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bf2q6" podStartSLOduration=67.948623237 podStartE2EDuration="1m9.473033733s" totalImagesPullingTime="1.524410496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:33.470884344 +0000 UTC m=+69.965079851" watchObservedRunningTime="2026-07-30 09:50:33.473033733 +0000 UTC m=+69.967229240"
Jul 30 09:50:33.508360 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:33.507312    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-888465b95-k24rp" podStartSLOduration=33.507293946 podStartE2EDuration="33.507293946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:33.50683661 +0000 UTC m=+70.001032130" watchObservedRunningTime="2026-07-30 09:50:33.507293946 +0000 UTC m=+70.001489451"
Jul 30 09:50:33.646223 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:33.646134    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:33.649837 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:33.649594    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:34.466454 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:34.466377    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:34.467714 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:34.467689    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-888465b95-k24rp"
Jul 30 09:50:35.108671 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:35.108628    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fhkxr\" (UID: \"8d349f02-e060-4dca-b9b9-44163a0fb935\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"
Jul 30 09:50:35.111435 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:35.111408    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d349f02-e060-4dca-b9b9-44163a0fb935-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-fhkxr\" (UID: \"8d349f02-e060-4dca-b9b9-44163a0fb935\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"
Jul 30 09:50:35.226060 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:35.226025    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-k54mz\""
Jul 30 09:50:35.234186 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:35.234155    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"
Jul 30 09:50:41.154660 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:41.154632    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr"]
Jul 30 09:50:43.967526 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:43.967492    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d349f02_e060_4dca_b9b9_44163a0fb935.slice/crio-663a5f80564fcec13fd07aff752e84e5ffc40492d50ce5eea68c8b8735f34a6f WatchSource:0}: Error finding container 663a5f80564fcec13fd07aff752e84e5ffc40492d50ce5eea68c8b8735f34a6f: Status 404 returned error can't find the container with id 663a5f80564fcec13fd07aff752e84e5ffc40492d50ce5eea68c8b8735f34a6f
Jul 30 09:50:44.514182 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.514055    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cr8nk" event={"ID":"ca650cbb-da82-43ee-b3fe-397ab13a2a25","Type":"ContainerStarted","Data":"0bbfd644230bdc59ea0e73c6bb47d046008d00a87fdc13b415b6dc9992c86b3b"}
Jul 30 09:50:44.516693 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.516651    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dfptt" event={"ID":"807d7a19-7b79-4963-a013-53d46d57b8a2","Type":"ContainerStarted","Data":"d72f34dba16012a6cf067e3fd76f9b523992f1c5963b1655896f5c1e2174fe3a"}
Jul 30 09:50:44.522010 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.521683    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988" event={"ID":"e64d61ea-9828-4770-827b-32abb45f45e5","Type":"ContainerStarted","Data":"3d79043e5f1aca448f668d474ac92ca45e2c72b0593e3b8ef48153b093b58644"}
Jul 30 09:50:44.522010 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.521728    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988" event={"ID":"e64d61ea-9828-4770-827b-32abb45f45e5","Type":"ContainerStarted","Data":"ffba350a371bf2efc9f68f762679eca3ced8a925bcb015ab9676bec6f98c9e61"}
Jul 30 09:50:44.524814 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.524785    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b" event={"ID":"31a81bf8-7bdf-4056-8a66-2de189f419a9","Type":"ContainerStarted","Data":"205e549f7897795130dc19cd2a90319ab9d90e9ae09c2a176b48603736ed73f2"}
Jul 30 09:50:44.524960 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.524950    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"
Jul 30 09:50:44.526618 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.526595    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b"
Jul 30 09:50:44.529444 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.528731    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf" event={"ID":"da4c528d-7534-4727-8ace-474996ab3454","Type":"ContainerStarted","Data":"73f93ec5cc275b3636971012a65efbbc691b825473ff50ef3788e3ee828e370e"}
Jul 30 09:50:44.531889 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.531859    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr" event={"ID":"8d349f02-e060-4dca-b9b9-44163a0fb935","Type":"ContainerStarted","Data":"663a5f80564fcec13fd07aff752e84e5ffc40492d50ce5eea68c8b8735f34a6f"}
Jul 30 09:50:44.533670 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.533649    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-84b5h" event={"ID":"3c1a1aad-170c-4e52-8f7d-e763663b81e4","Type":"ContainerStarted","Data":"13620e0000580d153b454d2a1273511b0c035931850d31258a95a53e3340cb1d"}
Jul 30 09:50:44.535315 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.535295    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-84b5h"
Jul 30 09:50:44.535793 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.535607    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dfptt" podStartSLOduration=67.375633955 podStartE2EDuration="1m20.53559008s" totalImagesPullingTime="13.159956125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 09:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:44.534366229 +0000 UTC m=+81.028561736" watchObservedRunningTime="2026-07-30 09:50:44.53559008 +0000 UTC m=+81.029785587"
Jul 30 09:50:44.536635 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.536604    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n" event={"ID":"d1b2a919-5411-474f-ad20-251fe1d2501a","Type":"ContainerStarted","Data":"39b64a0611318ff7e557fbb0291b17d01de1395a37db1ad4572226d59de6db17"}
Jul 30 09:50:44.552273 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.552222    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-84b5h" podStartSLOduration=36.356598196 podStartE2EDuration="47.55220409s" totalImagesPullingTime="11.195605894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:49:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:44.551390219 +0000 UTC m=+81.045585726" watchObservedRunningTime="2026-07-30 09:50:44.55220409 +0000 UTC m=+81.046399596"
Jul 30 09:50:44.571735 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.571688    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-mz9gf" podStartSLOduration=35.202367654 podStartE2EDuration="44.571670545s" totalImagesPullingTime="9.369302891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:44.570512691 +0000 UTC m=+81.064708198" watchObservedRunningTime="2026-07-30 09:50:44.571670545 +0000 UTC m=+81.065866042"
Jul 30 09:50:44.596555 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.596482    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-764f8fcd77-r5f7b" podStartSLOduration=1.359866844 podStartE2EDuration="12.596459328s" totalImagesPullingTime="11.236592484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:44.595582917 +0000 UTC m=+81.089778424" watchObservedRunningTime="2026-07-30 09:50:44.596459328 +0000 UTC m=+81.090654834"
Jul 30 09:50:44.614438 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:44.612972    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db4cc4b7b-t9j4n" podStartSLOduration=1.357798477 podStartE2EDuration="12.61293649s" totalImagesPullingTime="11.255138013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:44.611842425 +0000 UTC m=+81.106037931" watchObservedRunningTime="2026-07-30 09:50:44.61293649 +0000 UTC m=+81.107131996"
Jul 30 09:50:45.545680 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:45.545641    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988" event={"ID":"e64d61ea-9828-4770-827b-32abb45f45e5","Type":"ContainerStarted","Data":"97c05f2ed66405fb3968cd687a046c6e5664310327a1af2aae9ff7eb57c0d5a3"}
Jul 30 09:50:45.548636 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:45.548602    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-84b5h" event={"ID":"3c1a1aad-170c-4e52-8f7d-e763663b81e4","Type":"ContainerStarted","Data":"6ff38620ae612c815dd20c37da1ce4c4198897fc1becf1aa63a9b9470eb40338"}
Jul 30 09:50:45.567361 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:45.567274    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78fd754fbb-5q988" podStartSLOduration=2.350382297 podStartE2EDuration="13.567255796s" totalImagesPullingTime="11.216873499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:45.565580207 +0000 UTC m=+82.059775713" watchObservedRunningTime="2026-07-30 09:50:45.567255796 +0000 UTC m=+82.061451362"
Jul 30 09:50:46.058199 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:46.058054    2567 scope.go:117] "RemoveContainer" containerID="91116bcc022445da3513f72e9b41661ca25476bbe99307fad052abee0fca233e"
Jul 30 09:50:46.554670 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:46.554639    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 09:50:46.555121 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:46.554776    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c" event={"ID":"97e78a90-a448-4e2d-bdf4-5b21892bb617","Type":"ContainerStarted","Data":"861da0de5f6fb5d75ae7884c483770f6c802e2e0b11900a0e3f6da074b1e82e9"}
Jul 30 09:50:46.555314 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:46.555267    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:46.557001 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:46.556972    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr" event={"ID":"8d349f02-e060-4dca-b9b9-44163a0fb935","Type":"ContainerStarted","Data":"b9281e2d49e87ce79dfdda7b335d7751d6cb2366a36a88a620a98513e0a5f99f"}
Jul 30 09:50:46.561080 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:46.561056    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c"
Jul 30 09:50:46.572727 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:46.572665    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-hw45c" podStartSLOduration=39.376386725 podStartE2EDuration="46.572645333s" totalImagesPullingTime="7.196258608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:46.572043783 +0000 UTC m=+83.066239289" watchObservedRunningTime="2026-07-30 09:50:46.572645333 +0000 UTC m=+83.066840854"
Jul 30 09:50:46.620918 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:46.620813    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-fhkxr" podStartSLOduration=42.090500636 podStartE2EDuration="43.620792287s" totalImagesPullingTime="1.530291651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:46.619699092 +0000 UTC m=+83.113894599" watchObservedRunningTime="2026-07-30 09:50:46.620792287 +0000 UTC m=+83.114987794"
Jul 30 09:50:48.568024 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:48.567931    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cr8nk" event={"ID":"ca650cbb-da82-43ee-b3fe-397ab13a2a25","Type":"ContainerStarted","Data":"2f1b96465fd7ad86a2556eb12033c625e7236a6469bef6beb4bdfcad551be619"}
Jul 30 09:50:48.587422 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:48.587370    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cr8nk" podStartSLOduration=17.426857306 podStartE2EDuration="39.58732772s" totalImagesPullingTime="22.160470414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 09:50:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:50:48.587082329 +0000 UTC m=+85.081277834" watchObservedRunningTime="2026-07-30 09:50:48.58732772 +0000 UTC m=+85.081523226"
Jul 30 09:50:49.969516 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:49.969480    2567 patch_prober.go:28] interesting pod/image-registry-598bfb974b-kzqfn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 30 09:50:49.969891 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:49.969537    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-598bfb974b-kzqfn" podUID="2f923caa-9667-4c0d-8083-47944d143e9d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 09:50:52.446588 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:52.446555    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:50:55.276048 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:55.276012    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-598bfb974b-kzqfn"]
Jul 30 09:50:55.559254 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:55.559178    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-84b5h"
Jul 30 09:50:57.325505 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.325466    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ss6kg"]
Jul 30 09:50:57.328997 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.328977    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.335237 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.335212    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jfvzg\""
Jul 30 09:50:57.335415 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.335249    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 09:50:57.335415 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.335266    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 09:50:57.336827 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.336802    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 09:50:57.336937 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.336802    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 09:50:57.409177 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.409139    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-root\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.409382 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.409181    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-tls\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.409382 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.409207    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.409382 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.409272    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-sys\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.409382 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.409324    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-wtmp\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.409592 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.409452    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwg9c\" (UniqueName: \"kubernetes.io/projected/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-kube-api-access-rwg9c\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.409592 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.409489    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-metrics-client-ca\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.409592 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.409574    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-accelerators-collector-config\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.409702 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.409607    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-textfile\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.510274 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.510231    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-wtmp\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.510487 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.510353    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwg9c\" (UniqueName: \"kubernetes.io/projected/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-kube-api-access-rwg9c\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.510487 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.510400    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-metrics-client-ca\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.510487 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.510437    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-wtmp\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.510487 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.510448    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-accelerators-collector-config\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.510736 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.510511    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-textfile\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.510736 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.510540    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-root\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.510736 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.510559    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-tls\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.510736 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.510578    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.510736 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.510603    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-sys\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.510736 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.510626    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-root\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.510736 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.510688    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-sys\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.511001 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.510888    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-textfile\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.511096 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.511073    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-metrics-client-ca\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.511160 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.511141    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-accelerators-collector-config\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.513038 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.513009    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.513140 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.513039    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-node-exporter-tls\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.521351 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.521318    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwg9c\" (UniqueName: \"kubernetes.io/projected/dc2ff597-4391-4730-a8eb-ba4bfabf04ee-kube-api-access-rwg9c\") pod \"node-exporter-ss6kg\" (UID: \"dc2ff597-4391-4730-a8eb-ba4bfabf04ee\") " pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.640370 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:57.640320    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ss6kg"
Jul 30 09:50:57.652545 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:50:57.652517    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc2ff597_4391_4730_a8eb_ba4bfabf04ee.slice/crio-08e1058828901dfb9cce74a9a40ee0ac85d87cb73acc37b5968b7f5e581bb25d WatchSource:0}: Error finding container 08e1058828901dfb9cce74a9a40ee0ac85d87cb73acc37b5968b7f5e581bb25d: Status 404 returned error can't find the container with id 08e1058828901dfb9cce74a9a40ee0ac85d87cb73acc37b5968b7f5e581bb25d
Jul 30 09:50:58.612167 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:58.612133    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ss6kg" event={"ID":"dc2ff597-4391-4730-a8eb-ba4bfabf04ee","Type":"ContainerStarted","Data":"08e1058828901dfb9cce74a9a40ee0ac85d87cb73acc37b5968b7f5e581bb25d"}
Jul 30 09:50:59.617758 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:59.617726    2567 generic.go:356] "Generic (PLEG): container finished" podID="dc2ff597-4391-4730-a8eb-ba4bfabf04ee" containerID="5bb6aa8d419f67ee7ea89b74c38d6447c8e78386681c9861a43c3cfbc20ae9f3" exitCode=0
Jul 30 09:50:59.618181 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:50:59.617805    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ss6kg" event={"ID":"dc2ff597-4391-4730-a8eb-ba4bfabf04ee","Type":"ContainerDied","Data":"5bb6aa8d419f67ee7ea89b74c38d6447c8e78386681c9861a43c3cfbc20ae9f3"}
Jul 30 09:51:00.624251 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:00.624211    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ss6kg" event={"ID":"dc2ff597-4391-4730-a8eb-ba4bfabf04ee","Type":"ContainerStarted","Data":"ed84ea6741e30352de09a71038dfbdf2154f8ac3eb6dd0c8fa9553cfb3d984d2"}
Jul 30 09:51:00.624679 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:00.624293    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ss6kg" event={"ID":"dc2ff597-4391-4730-a8eb-ba4bfabf04ee","Type":"ContainerStarted","Data":"5da9cb54e292722f9cbe8788f92410e9adac241800acc2a0cfe80aa5f8f26e32"}
Jul 30 09:51:00.648108 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:00.648056    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ss6kg" podStartSLOduration=2.603447989 podStartE2EDuration="3.648040247s" totalImagesPullingTime="979.211948ms" totalInitContainerRuntime="65.38031ms" isStatefulPod=true podCreationTimestamp="2026-07-30 09:50:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:51:00.647512406 +0000 UTC m=+97.141707930" watchObservedRunningTime="2026-07-30 09:51:00.648040247 +0000 UTC m=+97.142235753"
Jul 30 09:51:02.445548 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:02.445515    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gxth2"
Jul 30 09:51:20.294827 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.294788    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-598bfb974b-kzqfn" podUID="2f923caa-9667-4c0d-8083-47944d143e9d" containerName="registry" containerID="cri-o://cf11c60bb8e7ca7ecd8ffb8251cd8afaf62c891adbbcf372f05ccddec1bd1e5d" gracePeriod=30
Jul 30 09:51:20.567137 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.567054    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:51:20.703935 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.703905    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccdsq\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-kube-api-access-ccdsq\") pod \"2f923caa-9667-4c0d-8083-47944d143e9d\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") "
Jul 30 09:51:20.704113 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.703963    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f923caa-9667-4c0d-8083-47944d143e9d-registry-certificates\") pod \"2f923caa-9667-4c0d-8083-47944d143e9d\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") "
Jul 30 09:51:20.704113 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.704009    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f923caa-9667-4c0d-8083-47944d143e9d-installation-pull-secrets\") pod \"2f923caa-9667-4c0d-8083-47944d143e9d\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") "
Jul 30 09:51:20.704113 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.704032    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-bound-sa-token\") pod \"2f923caa-9667-4c0d-8083-47944d143e9d\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") "
Jul 30 09:51:20.704113 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.704078    2567 generic.go:356] "Generic (PLEG): container finished" podID="2f923caa-9667-4c0d-8083-47944d143e9d" containerID="cf11c60bb8e7ca7ecd8ffb8251cd8afaf62c891adbbcf372f05ccddec1bd1e5d" exitCode=0
Jul 30 09:51:20.704373 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.704118    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-598bfb974b-kzqfn" event={"ID":"2f923caa-9667-4c0d-8083-47944d143e9d","Type":"ContainerDied","Data":"cf11c60bb8e7ca7ecd8ffb8251cd8afaf62c891adbbcf372f05ccddec1bd1e5d"}
Jul 30 09:51:20.704373 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.704148    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f923caa-9667-4c0d-8083-47944d143e9d-image-registry-private-configuration\") pod \"2f923caa-9667-4c0d-8083-47944d143e9d\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") "
Jul 30 09:51:20.704373 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.704173    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-598bfb974b-kzqfn"
Jul 30 09:51:20.704373 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.704182    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-598bfb974b-kzqfn" event={"ID":"2f923caa-9667-4c0d-8083-47944d143e9d","Type":"ContainerDied","Data":"61ea1f1b118c90f88b4577f97d7dd2f85ec7d203361181f95f726f488566b4d8"}
Jul 30 09:51:20.704373 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.704184    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f923caa-9667-4c0d-8083-47944d143e9d-ca-trust-extracted\") pod \"2f923caa-9667-4c0d-8083-47944d143e9d\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") "
Jul 30 09:51:20.704373 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.704270    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls\") pod \"2f923caa-9667-4c0d-8083-47944d143e9d\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") "
Jul 30 09:51:20.704691 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.704375    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f923caa-9667-4c0d-8083-47944d143e9d-trusted-ca\") pod \"2f923caa-9667-4c0d-8083-47944d143e9d\" (UID: \"2f923caa-9667-4c0d-8083-47944d143e9d\") "
Jul 30 09:51:20.704691 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.704409    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f923caa-9667-4c0d-8083-47944d143e9d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2f923caa-9667-4c0d-8083-47944d143e9d" (UID: "2f923caa-9667-4c0d-8083-47944d143e9d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 09:51:20.704691 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.704522    2567 scope.go:117] "RemoveContainer" containerID="cf11c60bb8e7ca7ecd8ffb8251cd8afaf62c891adbbcf372f05ccddec1bd1e5d"
Jul 30 09:51:20.704691 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.704619    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2f923caa-9667-4c0d-8083-47944d143e9d-registry-certificates\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 09:51:20.705666 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.705585    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f923caa-9667-4c0d-8083-47944d143e9d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2f923caa-9667-4c0d-8083-47944d143e9d" (UID: "2f923caa-9667-4c0d-8083-47944d143e9d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 09:51:20.707150 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.707083    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f923caa-9667-4c0d-8083-47944d143e9d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2f923caa-9667-4c0d-8083-47944d143e9d" (UID: "2f923caa-9667-4c0d-8083-47944d143e9d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 09:51:20.707150 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.707097    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-kube-api-access-ccdsq" (OuterVolumeSpecName: "kube-api-access-ccdsq") pod "2f923caa-9667-4c0d-8083-47944d143e9d" (UID: "2f923caa-9667-4c0d-8083-47944d143e9d"). InnerVolumeSpecName "kube-api-access-ccdsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 09:51:20.707349 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.707242    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2f923caa-9667-4c0d-8083-47944d143e9d" (UID: "2f923caa-9667-4c0d-8083-47944d143e9d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 09:51:20.707539 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.707510    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f923caa-9667-4c0d-8083-47944d143e9d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2f923caa-9667-4c0d-8083-47944d143e9d" (UID: "2f923caa-9667-4c0d-8083-47944d143e9d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 09:51:20.707539 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.707521    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2f923caa-9667-4c0d-8083-47944d143e9d" (UID: "2f923caa-9667-4c0d-8083-47944d143e9d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 09:51:20.713757 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.713729    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f923caa-9667-4c0d-8083-47944d143e9d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2f923caa-9667-4c0d-8083-47944d143e9d" (UID: "2f923caa-9667-4c0d-8083-47944d143e9d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 09:51:20.723318 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.723297    2567 scope.go:117] "RemoveContainer" containerID="cf11c60bb8e7ca7ecd8ffb8251cd8afaf62c891adbbcf372f05ccddec1bd1e5d"
Jul 30 09:51:20.723658 ip-10-0-128-146 kubenswrapper[2567]: E0730 09:51:20.723636    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf11c60bb8e7ca7ecd8ffb8251cd8afaf62c891adbbcf372f05ccddec1bd1e5d\": container with ID starting with cf11c60bb8e7ca7ecd8ffb8251cd8afaf62c891adbbcf372f05ccddec1bd1e5d not found: ID does not exist" containerID="cf11c60bb8e7ca7ecd8ffb8251cd8afaf62c891adbbcf372f05ccddec1bd1e5d"
Jul 30 09:51:20.723705 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.723669    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf11c60bb8e7ca7ecd8ffb8251cd8afaf62c891adbbcf372f05ccddec1bd1e5d"} err="failed to get container status \"cf11c60bb8e7ca7ecd8ffb8251cd8afaf62c891adbbcf372f05ccddec1bd1e5d\": rpc error: code = NotFound desc = could not find container \"cf11c60bb8e7ca7ecd8ffb8251cd8afaf62c891adbbcf372f05ccddec1bd1e5d\": container with ID starting with cf11c60bb8e7ca7ecd8ffb8251cd8afaf62c891adbbcf372f05ccddec1bd1e5d not found: ID does not exist"
Jul 30 09:51:20.805193 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.805148    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2f923caa-9667-4c0d-8083-47944d143e9d-installation-pull-secrets\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 09:51:20.805193 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.805185    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-bound-sa-token\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 09:51:20.805193 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.805195    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2f923caa-9667-4c0d-8083-47944d143e9d-image-registry-private-configuration\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 09:51:20.805193 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.805204    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2f923caa-9667-4c0d-8083-47944d143e9d-ca-trust-extracted\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 09:51:20.805499 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.805214    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-registry-tls\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 09:51:20.805499 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.805221    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f923caa-9667-4c0d-8083-47944d143e9d-trusted-ca\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 09:51:20.805499 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:20.805229    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccdsq\" (UniqueName: \"kubernetes.io/projected/2f923caa-9667-4c0d-8083-47944d143e9d-kube-api-access-ccdsq\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 09:51:21.029104 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:21.029070    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-598bfb974b-kzqfn"]
Jul 30 09:51:21.036394 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:21.036366    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-598bfb974b-kzqfn"]
Jul 30 09:51:22.062055 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:22.062017    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f923caa-9667-4c0d-8083-47944d143e9d" path="/var/lib/kubelet/pods/2f923caa-9667-4c0d-8083-47944d143e9d/volumes"
Jul 30 09:51:24.721700 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:24.721665    2567 generic.go:356] "Generic (PLEG): container finished" podID="a79aad4c-afde-4ab7-b54a-4fd145a20252" containerID="314b34a3a7e94d78896ee1b5e66cca094028cdd162765b44d9daf8e6f20be3a4" exitCode=0
Jul 30 09:51:24.722068 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:24.721738    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r" event={"ID":"a79aad4c-afde-4ab7-b54a-4fd145a20252","Type":"ContainerDied","Data":"314b34a3a7e94d78896ee1b5e66cca094028cdd162765b44d9daf8e6f20be3a4"}
Jul 30 09:51:24.722120 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:24.722103    2567 scope.go:117] "RemoveContainer" containerID="314b34a3a7e94d78896ee1b5e66cca094028cdd162765b44d9daf8e6f20be3a4"
Jul 30 09:51:25.727414 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:25.727381    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r" event={"ID":"a79aad4c-afde-4ab7-b54a-4fd145a20252","Type":"ContainerStarted","Data":"5af7f9bc99228955e4bbb79c1b21b1af9e44dfffbdd322fc5b08ce196fc1b540"}
Jul 30 09:51:29.748260 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:29.748222    2567 generic.go:356] "Generic (PLEG): container finished" podID="24fde817-36dc-436d-8706-47bdce301175" containerID="93dfd7be50a1b0a25fa879744304d404edcd6cb014c76a3bafca9c25a9715d79" exitCode=0
Jul 30 09:51:29.748704 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:29.748281    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78" event={"ID":"24fde817-36dc-436d-8706-47bdce301175","Type":"ContainerDied","Data":"93dfd7be50a1b0a25fa879744304d404edcd6cb014c76a3bafca9c25a9715d79"}
Jul 30 09:51:29.748704 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:29.748607    2567 scope.go:117] "RemoveContainer" containerID="93dfd7be50a1b0a25fa879744304d404edcd6cb014c76a3bafca9c25a9715d79"
Jul 30 09:51:30.753762 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:30.753727    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-stc78" event={"ID":"24fde817-36dc-436d-8706-47bdce301175","Type":"ContainerStarted","Data":"83da2a54e10375110d50065fed3e73f439715f4cf62d9625df137d80365cd69b"}
Jul 30 09:51:44.807869 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:44.807829    2567 generic.go:356] "Generic (PLEG): container finished" podID="353dc841-0a51-4587-b4a9-6a4955ad1a12" containerID="c5646de131751c7fd7fe8c0132ecf144a5a93ba0eb17732540a99757d7493ae0" exitCode=0
Jul 30 09:51:44.808261 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:44.807899    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-p8h62" event={"ID":"353dc841-0a51-4587-b4a9-6a4955ad1a12","Type":"ContainerDied","Data":"c5646de131751c7fd7fe8c0132ecf144a5a93ba0eb17732540a99757d7493ae0"}
Jul 30 09:51:44.808261 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:44.808245    2567 scope.go:117] "RemoveContainer" containerID="c5646de131751c7fd7fe8c0132ecf144a5a93ba0eb17732540a99757d7493ae0"
Jul 30 09:51:45.814328 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:51:45.814296    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-p8h62" event={"ID":"353dc841-0a51-4587-b4a9-6a4955ad1a12","Type":"ContainerStarted","Data":"47854abc5049d1a4e70d5d76e8bba5a549e45adf826ac53e1b17701252fc395b"}
Jul 30 09:54:23.859360 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:54:23.859303    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 09:54:23.863315 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:54:23.863285    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 09:54:23.865379 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:54:23.865355    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 09:54:23.870369 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:54:23.870324    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 09:54:23.871450 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:54:23.871432    2567 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 09:55:58.546002 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.545965    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"]
Jul 30 09:55:58.546510 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.546274    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f923caa-9667-4c0d-8083-47944d143e9d" containerName="registry"
Jul 30 09:55:58.546510 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.546295    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f923caa-9667-4c0d-8083-47944d143e9d" containerName="registry"
Jul 30 09:55:58.546510 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.546373    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f923caa-9667-4c0d-8083-47944d143e9d" containerName="registry"
Jul 30 09:55:58.548706 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.548692    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.552677 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.552653    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 09:55:58.552853 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.552689    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nzz9n\""
Jul 30 09:55:58.552853 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.552735    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 09:55:58.552992 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.552977    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 09:55:58.553200 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.553186    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 09:55:58.573041 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.573013    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"]
Jul 30 09:55:58.616895 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.616859    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.617087 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.616964    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.617087 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.616998    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-manifests\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.617087 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.617044    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x686\" (UniqueName: \"kubernetes.io/projected/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-kube-api-access-9x686\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.617087 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.617079    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-tmp\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.718424 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.718371    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x686\" (UniqueName: \"kubernetes.io/projected/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-kube-api-access-9x686\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.718424 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.718429    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-tmp\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.718733 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.718477    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.718733 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.718533    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.718733 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.718551    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-manifests\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.718955 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.718928    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-manifests\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.720840 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.720807    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-tmp\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.720975 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.720956    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.721045 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.721025    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.727797 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.727769    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x686\" (UniqueName: \"kubernetes.io/projected/ba51c3d1-3817-49b8-a583-3bdc4dd70c5c-kube-api-access-9x686\") pod \"opendatahub-operator-controller-manager-5d74749788-q6kvh\" (UID: \"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.859727 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.859624    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:55:58.999948 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:58.999915    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"]
Jul 30 09:55:59.003439 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:55:59.003411    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba51c3d1_3817_49b8_a583_3bdc4dd70c5c.slice/crio-287a7b7b2a15545e7fb6e87b8cbc5d195b267c77a95862373557e6441399b90d WatchSource:0}: Error finding container 287a7b7b2a15545e7fb6e87b8cbc5d195b267c77a95862373557e6441399b90d: Status 404 returned error can't find the container with id 287a7b7b2a15545e7fb6e87b8cbc5d195b267c77a95862373557e6441399b90d
Jul 30 09:55:59.005160 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:59.005140    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 09:55:59.785309 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:55:59.785272    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh" event={"ID":"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c","Type":"ContainerStarted","Data":"287a7b7b2a15545e7fb6e87b8cbc5d195b267c77a95862373557e6441399b90d"}
Jul 30 09:56:02.801928 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:02.801889    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh" event={"ID":"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c","Type":"ContainerStarted","Data":"054e9d413dbfa1eb58895a0c1f9695bbbb0768a4269262c3a93875c6ba1f7802"}
Jul 30 09:56:03.808248 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:03.808213    2567 generic.go:356] "Generic (PLEG): container finished" podID="ba51c3d1-3817-49b8-a583-3bdc4dd70c5c" containerID="054e9d413dbfa1eb58895a0c1f9695bbbb0768a4269262c3a93875c6ba1f7802" exitCode=0
Jul 30 09:56:03.808653 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:03.808260    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh" event={"ID":"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c","Type":"ContainerDied","Data":"054e9d413dbfa1eb58895a0c1f9695bbbb0768a4269262c3a93875c6ba1f7802"}
Jul 30 09:56:05.822273 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:05.822173    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh" event={"ID":"ba51c3d1-3817-49b8-a583-3bdc4dd70c5c","Type":"ContainerStarted","Data":"1acf99c879131beb4f1c1802c42384aa8a8dbf47b8df478a380ba7904e0b6961"}
Jul 30 09:56:05.822273 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:05.822256    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:56:05.844379 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:05.844280    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh" podStartSLOduration=2.6336330329999997 podStartE2EDuration="7.844259452s" totalImagesPullingTime="4.13798992s" totalInitContainerRuntime="1.072636499s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:55:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:56:05.84228121 +0000 UTC m=+402.336476716" watchObservedRunningTime="2026-07-30 09:56:05.844259452 +0000 UTC m=+402.338454962"
Jul 30 09:56:16.828748 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:16.828715    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d74749788-q6kvh"
Jul 30 09:56:26.765819 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.765782    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"]
Jul 30 09:56:26.768997 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.768972    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:26.772242 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.772212    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 30 09:56:26.772494 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.772478    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 30 09:56:26.773258 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.773241    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 30 09:56:26.773524 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.773506    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-n8wmp\""
Jul 30 09:56:26.773614 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.773546    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 30 09:56:26.773614 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.773550    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 30 09:56:26.778515 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.778490    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"]
Jul 30 09:56:26.860824 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.860789    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fabd841c-cd79-4c2c-b74a-ca48acf65cdc-cert\") pod \"lws-controller-manager-75b6964665-2rpdh\" (UID: \"fabd841c-cd79-4c2c-b74a-ca48acf65cdc\") " pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:26.861004 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.860842    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fabd841c-cd79-4c2c-b74a-ca48acf65cdc-manager-config\") pod \"lws-controller-manager-75b6964665-2rpdh\" (UID: \"fabd841c-cd79-4c2c-b74a-ca48acf65cdc\") " pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:26.861004 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.860863    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fabd841c-cd79-4c2c-b74a-ca48acf65cdc-metrics-cert\") pod \"lws-controller-manager-75b6964665-2rpdh\" (UID: \"fabd841c-cd79-4c2c-b74a-ca48acf65cdc\") " pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:26.861004 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.860917    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24bgh\" (UniqueName: \"kubernetes.io/projected/fabd841c-cd79-4c2c-b74a-ca48acf65cdc-kube-api-access-24bgh\") pod \"lws-controller-manager-75b6964665-2rpdh\" (UID: \"fabd841c-cd79-4c2c-b74a-ca48acf65cdc\") " pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:26.961850 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.961815    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fabd841c-cd79-4c2c-b74a-ca48acf65cdc-cert\") pod \"lws-controller-manager-75b6964665-2rpdh\" (UID: \"fabd841c-cd79-4c2c-b74a-ca48acf65cdc\") " pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:26.962028 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.961862    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fabd841c-cd79-4c2c-b74a-ca48acf65cdc-manager-config\") pod \"lws-controller-manager-75b6964665-2rpdh\" (UID: \"fabd841c-cd79-4c2c-b74a-ca48acf65cdc\") " pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:26.962028 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.962025    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fabd841c-cd79-4c2c-b74a-ca48acf65cdc-metrics-cert\") pod \"lws-controller-manager-75b6964665-2rpdh\" (UID: \"fabd841c-cd79-4c2c-b74a-ca48acf65cdc\") " pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:26.962118 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.962047    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24bgh\" (UniqueName: \"kubernetes.io/projected/fabd841c-cd79-4c2c-b74a-ca48acf65cdc-kube-api-access-24bgh\") pod \"lws-controller-manager-75b6964665-2rpdh\" (UID: \"fabd841c-cd79-4c2c-b74a-ca48acf65cdc\") " pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:26.962686 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.962660    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fabd841c-cd79-4c2c-b74a-ca48acf65cdc-manager-config\") pod \"lws-controller-manager-75b6964665-2rpdh\" (UID: \"fabd841c-cd79-4c2c-b74a-ca48acf65cdc\") " pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:26.964351 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.964317    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fabd841c-cd79-4c2c-b74a-ca48acf65cdc-cert\") pod \"lws-controller-manager-75b6964665-2rpdh\" (UID: \"fabd841c-cd79-4c2c-b74a-ca48acf65cdc\") " pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:26.964519 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.964499    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fabd841c-cd79-4c2c-b74a-ca48acf65cdc-metrics-cert\") pod \"lws-controller-manager-75b6964665-2rpdh\" (UID: \"fabd841c-cd79-4c2c-b74a-ca48acf65cdc\") " pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:26.972316 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:26.972296    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24bgh\" (UniqueName: \"kubernetes.io/projected/fabd841c-cd79-4c2c-b74a-ca48acf65cdc-kube-api-access-24bgh\") pod \"lws-controller-manager-75b6964665-2rpdh\" (UID: \"fabd841c-cd79-4c2c-b74a-ca48acf65cdc\") " pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:27.079703 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:27.079612    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:27.219680 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:27.219616    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"]
Jul 30 09:56:27.221716 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:56:27.221686    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfabd841c_cd79_4c2c_b74a_ca48acf65cdc.slice/crio-3537a76d496fd30ec207ca7fe1d1989419714b5c1fd7c0e2f2e0b56a1aa0361d WatchSource:0}: Error finding container 3537a76d496fd30ec207ca7fe1d1989419714b5c1fd7c0e2f2e0b56a1aa0361d: Status 404 returned error can't find the container with id 3537a76d496fd30ec207ca7fe1d1989419714b5c1fd7c0e2f2e0b56a1aa0361d
Jul 30 09:56:27.914689 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:27.914651    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh" event={"ID":"fabd841c-cd79-4c2c-b74a-ca48acf65cdc","Type":"ContainerStarted","Data":"3537a76d496fd30ec207ca7fe1d1989419714b5c1fd7c0e2f2e0b56a1aa0361d"}
Jul 30 09:56:29.925814 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:29.925782    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh" event={"ID":"fabd841c-cd79-4c2c-b74a-ca48acf65cdc","Type":"ContainerStarted","Data":"29ac9ab4276fcfefeca7513dd94af1d275f3cc6d04e0917278271bfd195c2142"}
Jul 30 09:56:29.926203 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:29.925978    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:56:29.957127 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:29.957052    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh" podStartSLOduration=1.3344939679999999 podStartE2EDuration="3.957034767s" totalImagesPullingTime="2.622540799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:56:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:56:29.955228195 +0000 UTC m=+426.449423701" watchObservedRunningTime="2026-07-30 09:56:29.957034767 +0000 UTC m=+426.451230273"
Jul 30 09:56:31.216195 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.216156    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-72dg5"]
Jul 30 09:56:31.218479 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.218452    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-72dg5"
Jul 30 09:56:31.223343 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.223305    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 30 09:56:31.223493 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.223450    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf"]
Jul 30 09:56:31.223696 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.223677    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-nq78x\""
Jul 30 09:56:31.225525 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.225504    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf"
Jul 30 09:56:31.231096 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.231075    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 30 09:56:31.231825 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.231809    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-pfdl6\""
Jul 30 09:56:31.237175 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.237148    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-72dg5"]
Jul 30 09:56:31.240231 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.240210    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf"]
Jul 30 09:56:31.300376 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.300317    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d0a38f06-9a44-464b-acbd-6d51e7c2788b-cert\") pod \"kserve-controller-manager-d6f4b65b8-72dg5\" (UID: \"d0a38f06-9a44-464b-acbd-6d51e7c2788b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-72dg5"
Jul 30 09:56:31.300558 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.300395    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwck6\" (UniqueName: \"kubernetes.io/projected/ae848eb4-80e1-455f-b4b5-8aa109e02ad9-kube-api-access-wwck6\") pod \"llmisvc-controller-manager-777c6bd7d5-wtmnf\" (UID: \"ae848eb4-80e1-455f-b4b5-8aa109e02ad9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf"
Jul 30 09:56:31.300558 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.300436    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbbkg\" (UniqueName: \"kubernetes.io/projected/d0a38f06-9a44-464b-acbd-6d51e7c2788b-kube-api-access-wbbkg\") pod \"kserve-controller-manager-d6f4b65b8-72dg5\" (UID: \"d0a38f06-9a44-464b-acbd-6d51e7c2788b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-72dg5"
Jul 30 09:56:31.300558 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.300501    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae848eb4-80e1-455f-b4b5-8aa109e02ad9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wtmnf\" (UID: \"ae848eb4-80e1-455f-b4b5-8aa109e02ad9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf"
Jul 30 09:56:31.401276 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.401239    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae848eb4-80e1-455f-b4b5-8aa109e02ad9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wtmnf\" (UID: \"ae848eb4-80e1-455f-b4b5-8aa109e02ad9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf"
Jul 30 09:56:31.401489 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.401285    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d0a38f06-9a44-464b-acbd-6d51e7c2788b-cert\") pod \"kserve-controller-manager-d6f4b65b8-72dg5\" (UID: \"d0a38f06-9a44-464b-acbd-6d51e7c2788b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-72dg5"
Jul 30 09:56:31.401489 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.401317    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwck6\" (UniqueName: \"kubernetes.io/projected/ae848eb4-80e1-455f-b4b5-8aa109e02ad9-kube-api-access-wwck6\") pod \"llmisvc-controller-manager-777c6bd7d5-wtmnf\" (UID: \"ae848eb4-80e1-455f-b4b5-8aa109e02ad9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf"
Jul 30 09:56:31.401489 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.401372    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbbkg\" (UniqueName: \"kubernetes.io/projected/d0a38f06-9a44-464b-acbd-6d51e7c2788b-kube-api-access-wbbkg\") pod \"kserve-controller-manager-d6f4b65b8-72dg5\" (UID: \"d0a38f06-9a44-464b-acbd-6d51e7c2788b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-72dg5"
Jul 30 09:56:31.403763 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.403736    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d0a38f06-9a44-464b-acbd-6d51e7c2788b-cert\") pod \"kserve-controller-manager-d6f4b65b8-72dg5\" (UID: \"d0a38f06-9a44-464b-acbd-6d51e7c2788b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-72dg5"
Jul 30 09:56:31.403890 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.403764    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ae848eb4-80e1-455f-b4b5-8aa109e02ad9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wtmnf\" (UID: \"ae848eb4-80e1-455f-b4b5-8aa109e02ad9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf"
Jul 30 09:56:31.418830 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.418795    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbbkg\" (UniqueName: \"kubernetes.io/projected/d0a38f06-9a44-464b-acbd-6d51e7c2788b-kube-api-access-wbbkg\") pod \"kserve-controller-manager-d6f4b65b8-72dg5\" (UID: \"d0a38f06-9a44-464b-acbd-6d51e7c2788b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-72dg5"
Jul 30 09:56:31.419085 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.419064    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwck6\" (UniqueName: \"kubernetes.io/projected/ae848eb4-80e1-455f-b4b5-8aa109e02ad9-kube-api-access-wwck6\") pod \"llmisvc-controller-manager-777c6bd7d5-wtmnf\" (UID: \"ae848eb4-80e1-455f-b4b5-8aa109e02ad9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf"
Jul 30 09:56:31.530380 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.530257    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-72dg5"
Jul 30 09:56:31.537753 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.537727    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf"
Jul 30 09:56:31.690543 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.690514    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-72dg5"]
Jul 30 09:56:31.692542 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:56:31.692495    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0a38f06_9a44_464b_acbd_6d51e7c2788b.slice/crio-84040e6362ab3d9cb442ac60e36f8222d44821a651f2e9975cf39fcf22607cc2 WatchSource:0}: Error finding container 84040e6362ab3d9cb442ac60e36f8222d44821a651f2e9975cf39fcf22607cc2: Status 404 returned error can't find the container with id 84040e6362ab3d9cb442ac60e36f8222d44821a651f2e9975cf39fcf22607cc2
Jul 30 09:56:31.714429 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.714407    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf"]
Jul 30 09:56:31.716209 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:56:31.716181    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podae848eb4_80e1_455f_b4b5_8aa109e02ad9.slice/crio-fa245489492148450e3c5af01721e7259b9561f914a1b0be0ae10c297cc9ac2a WatchSource:0}: Error finding container fa245489492148450e3c5af01721e7259b9561f914a1b0be0ae10c297cc9ac2a: Status 404 returned error can't find the container with id fa245489492148450e3c5af01721e7259b9561f914a1b0be0ae10c297cc9ac2a
Jul 30 09:56:31.934887 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.934849    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf" event={"ID":"ae848eb4-80e1-455f-b4b5-8aa109e02ad9","Type":"ContainerStarted","Data":"fa245489492148450e3c5af01721e7259b9561f914a1b0be0ae10c297cc9ac2a"}
Jul 30 09:56:31.936059 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:31.936038    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-72dg5" event={"ID":"d0a38f06-9a44-464b-acbd-6d51e7c2788b","Type":"ContainerStarted","Data":"84040e6362ab3d9cb442ac60e36f8222d44821a651f2e9975cf39fcf22607cc2"}
Jul 30 09:56:35.955872 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:35.955831    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf" event={"ID":"ae848eb4-80e1-455f-b4b5-8aa109e02ad9","Type":"ContainerStarted","Data":"b55ea61d5021f6d3fd6a0f47e9230cbbca68e57290e188c1ff5d13473534934a"}
Jul 30 09:56:35.956388 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:35.955934    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf"
Jul 30 09:56:35.957669 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:35.957638    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-72dg5" event={"ID":"d0a38f06-9a44-464b-acbd-6d51e7c2788b","Type":"ContainerStarted","Data":"fd3919cb3b91bd0181e5e196d941880b74eb1380f8bc052436910ec0624f7650"}
Jul 30 09:56:35.957790 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:35.957705    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-72dg5"
Jul 30 09:56:35.972944 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:35.972891    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf" podStartSLOduration=1.371508401 podStartE2EDuration="4.97287287s" totalImagesPullingTime="3.601364469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:56:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:56:35.971427867 +0000 UTC m=+432.465623374" watchObservedRunningTime="2026-07-30 09:56:35.97287287 +0000 UTC m=+432.467068424"
Jul 30 09:56:35.988006 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:35.987949    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-72dg5" podStartSLOduration=1.367178026 podStartE2EDuration="4.987929637s" totalImagesPullingTime="3.620751611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:56:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:56:35.986933989 +0000 UTC m=+432.481129496" watchObservedRunningTime="2026-07-30 09:56:35.987929637 +0000 UTC m=+432.482125144"
Jul 30 09:56:40.932399 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:56:40.932364    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-75b6964665-2rpdh"
Jul 30 09:57:06.963590 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:57:06.963561    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wtmnf"
Jul 30 09:57:06.966465 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:57:06.966441    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-72dg5"
Jul 30 09:58:17.273212 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:17.273130    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68zjb"]
Jul 30 09:58:17.276404 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:17.276388    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68zjb"
Jul 30 09:58:17.280713 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:17.280692    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 09:58:17.281145 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:17.281127    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 09:58:17.281642 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:17.281627    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-r4vmp\""
Jul 30 09:58:17.292208 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:17.292181    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68zjb"]
Jul 30 09:58:17.373716 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:17.373678    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p59b8\" (UniqueName: \"kubernetes.io/projected/917861bf-8312-496d-93ea-76d8f0679218-kube-api-access-p59b8\") pod \"limitador-operator-controller-manager-859d7b78bd-68zjb\" (UID: \"917861bf-8312-496d-93ea-76d8f0679218\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68zjb"
Jul 30 09:58:17.474874 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:17.474813    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p59b8\" (UniqueName: \"kubernetes.io/projected/917861bf-8312-496d-93ea-76d8f0679218-kube-api-access-p59b8\") pod \"limitador-operator-controller-manager-859d7b78bd-68zjb\" (UID: \"917861bf-8312-496d-93ea-76d8f0679218\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68zjb"
Jul 30 09:58:17.483424 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:17.483400    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p59b8\" (UniqueName: \"kubernetes.io/projected/917861bf-8312-496d-93ea-76d8f0679218-kube-api-access-p59b8\") pod \"limitador-operator-controller-manager-859d7b78bd-68zjb\" (UID: \"917861bf-8312-496d-93ea-76d8f0679218\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68zjb"
Jul 30 09:58:17.586687 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:17.586598    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68zjb"
Jul 30 09:58:17.726465 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:58:17.726430    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod917861bf_8312_496d_93ea_76d8f0679218.slice/crio-a751217d31c386845970b3a18a01f254c5fc625f6922b49ae8de996608766e02 WatchSource:0}: Error finding container a751217d31c386845970b3a18a01f254c5fc625f6922b49ae8de996608766e02: Status 404 returned error can't find the container with id a751217d31c386845970b3a18a01f254c5fc625f6922b49ae8de996608766e02
Jul 30 09:58:17.728800 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:17.728745    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68zjb"]
Jul 30 09:58:18.389677 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:18.389645    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68zjb" event={"ID":"917861bf-8312-496d-93ea-76d8f0679218","Type":"ContainerStarted","Data":"a751217d31c386845970b3a18a01f254c5fc625f6922b49ae8de996608766e02"}
Jul 30 09:58:20.401152 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:20.401122    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68zjb" event={"ID":"917861bf-8312-496d-93ea-76d8f0679218","Type":"ContainerStarted","Data":"9a9b4b7713ff289993f82f4b9788d1f3485301f9ca5afd2b8259df02e0bac789"}
Jul 30 09:58:20.401558 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:20.401243    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68zjb"
Jul 30 09:58:20.419345 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:20.419291    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68zjb" podStartSLOduration=1.69617526 podStartE2EDuration="3.419275834s" totalImagesPullingTime="1.723100574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:58:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:58:20.418413153 +0000 UTC m=+536.912608660" watchObservedRunningTime="2026-07-30 09:58:20.419275834 +0000 UTC m=+536.913471381"
Jul 30 09:58:31.407426 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:58:31.407394    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-68zjb"
Jul 30 09:59:00.403401 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:00.403369    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 09:59:00.410090 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:00.410065    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"
Jul 30 09:59:00.412772 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:00.412746    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vgwcl\""
Jul 30 09:59:00.412900 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:00.412782    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 09:59:00.418210 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:00.418187    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 09:59:00.456426 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:00.456385    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 09:59:00.542916 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:00.542881    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg8zh\" (UniqueName: \"kubernetes.io/projected/12ab571e-0521-4dc0-98bd-5ac1f7215401-kube-api-access-lg8zh\") pod \"limitador-limitador-54d6d988d5-8xjwm\" (UID: \"12ab571e-0521-4dc0-98bd-5ac1f7215401\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"
Jul 30 09:59:00.542916 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:00.542925    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/12ab571e-0521-4dc0-98bd-5ac1f7215401-config-file\") pod \"limitador-limitador-54d6d988d5-8xjwm\" (UID: \"12ab571e-0521-4dc0-98bd-5ac1f7215401\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"
Jul 30 09:59:00.643677 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:00.643635    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg8zh\" (UniqueName: \"kubernetes.io/projected/12ab571e-0521-4dc0-98bd-5ac1f7215401-kube-api-access-lg8zh\") pod \"limitador-limitador-54d6d988d5-8xjwm\" (UID: \"12ab571e-0521-4dc0-98bd-5ac1f7215401\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"
Jul 30 09:59:00.643677 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:00.643680    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/12ab571e-0521-4dc0-98bd-5ac1f7215401-config-file\") pod \"limitador-limitador-54d6d988d5-8xjwm\" (UID: \"12ab571e-0521-4dc0-98bd-5ac1f7215401\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"
Jul 30 09:59:00.644269 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:00.644250    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/12ab571e-0521-4dc0-98bd-5ac1f7215401-config-file\") pod \"limitador-limitador-54d6d988d5-8xjwm\" (UID: \"12ab571e-0521-4dc0-98bd-5ac1f7215401\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"
Jul 30 09:59:00.655895 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:00.655843    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg8zh\" (UniqueName: \"kubernetes.io/projected/12ab571e-0521-4dc0-98bd-5ac1f7215401-kube-api-access-lg8zh\") pod \"limitador-limitador-54d6d988d5-8xjwm\" (UID: \"12ab571e-0521-4dc0-98bd-5ac1f7215401\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"
Jul 30 09:59:00.720508 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:00.720470    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"
Jul 30 09:59:00.876547 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:00.876471    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 09:59:00.879233 ip-10-0-128-146 kubenswrapper[2567]: W0730 09:59:00.879207    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ab571e_0521_4dc0_98bd_5ac1f7215401.slice/crio-be34cdfba4c33015d732eddcbaa864433d1dd949b08bd6ded6bb4632b1c0cf0f WatchSource:0}: Error finding container be34cdfba4c33015d732eddcbaa864433d1dd949b08bd6ded6bb4632b1c0cf0f: Status 404 returned error can't find the container with id be34cdfba4c33015d732eddcbaa864433d1dd949b08bd6ded6bb4632b1c0cf0f
Jul 30 09:59:01.591490 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:01.591383    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8xjwm" event={"ID":"12ab571e-0521-4dc0-98bd-5ac1f7215401","Type":"ContainerStarted","Data":"be34cdfba4c33015d732eddcbaa864433d1dd949b08bd6ded6bb4632b1c0cf0f"}
Jul 30 09:59:03.603127 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:03.603086    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8xjwm" event={"ID":"12ab571e-0521-4dc0-98bd-5ac1f7215401","Type":"ContainerStarted","Data":"6241a512ddefc24f4ad01f5009472f0948220c9080132a779e63fccbeb46d4f8"}
Jul 30 09:59:03.603559 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:03.603196    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"
Jul 30 09:59:03.624587 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:03.624534    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-8xjwm" podStartSLOduration=1.046270755 podStartE2EDuration="3.624516973s" totalImagesPullingTime="2.578246218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 09:59:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 09:59:03.623477594 +0000 UTC m=+580.117673101" watchObservedRunningTime="2026-07-30 09:59:03.624516973 +0000 UTC m=+580.118712478"
Jul 30 09:59:14.608260 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:14.608229    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"
Jul 30 09:59:23.888379 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:23.888345    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 09:59:23.889912 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:23.889887    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 09:59:23.894299 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:23.894278    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 09:59:23.895984 ip-10-0-128-146 kubenswrapper[2567]: I0730 09:59:23.895960    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:00:15.119327 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:00:15.119287    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:00:44.104010 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:00:44.103979    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:00:50.999645 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:00:50.999605    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:01:12.592235 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:12.592200    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:01:20.006587 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:20.006504    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:01:27.511724 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.511685    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"]
Jul 30 10:01:27.515226 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.515206    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.518012 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.517990    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 30 10:01:27.518137 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.518014    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 10:01:27.519002 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.518981    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mcpsp\""
Jul 30 10:01:27.519119 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.519034    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 10:01:27.524112 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.524092    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"]
Jul 30 10:01:27.674713 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.674669    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grvh4\" (UniqueName: \"kubernetes.io/projected/aff02424-d441-4ba2-9304-0745ff5972b0-kube-api-access-grvh4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.674713 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.674715    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aff02424-d441-4ba2-9304-0745ff5972b0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.674917 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.674765    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aff02424-d441-4ba2-9304-0745ff5972b0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.674917 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.674788    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aff02424-d441-4ba2-9304-0745ff5972b0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.674917 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.674868    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aff02424-d441-4ba2-9304-0745ff5972b0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.674917 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.674900    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aff02424-d441-4ba2-9304-0745ff5972b0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.776092 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.776004    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aff02424-d441-4ba2-9304-0745ff5972b0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.776092 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.776050    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aff02424-d441-4ba2-9304-0745ff5972b0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.776092 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.776079    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aff02424-d441-4ba2-9304-0745ff5972b0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.776092 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.776097    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aff02424-d441-4ba2-9304-0745ff5972b0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.776424 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.776135    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grvh4\" (UniqueName: \"kubernetes.io/projected/aff02424-d441-4ba2-9304-0745ff5972b0-kube-api-access-grvh4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.776424 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.776154    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aff02424-d441-4ba2-9304-0745ff5972b0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.776541 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.776513    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aff02424-d441-4ba2-9304-0745ff5972b0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.776597 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.776558    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aff02424-d441-4ba2-9304-0745ff5972b0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.776651 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.776609    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aff02424-d441-4ba2-9304-0745ff5972b0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.778550 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.778522    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aff02424-d441-4ba2-9304-0745ff5972b0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.778661 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.778622    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aff02424-d441-4ba2-9304-0745ff5972b0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.785049 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.785022    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grvh4\" (UniqueName: \"kubernetes.io/projected/aff02424-d441-4ba2-9304-0745ff5972b0-kube-api-access-grvh4\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-swzr6\" (UID: \"aff02424-d441-4ba2-9304-0745ff5972b0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.826290 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.826250    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:27.962281 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.962259    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"]
Jul 30 10:01:27.965410 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:01:27.965379    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff02424_d441_4ba2_9304_0745ff5972b0.slice/crio-b787d7587d038ae35b5eab6546ec01b5b1752182c3e76c7655f5ba53819d1203 WatchSource:0}: Error finding container b787d7587d038ae35b5eab6546ec01b5b1752182c3e76c7655f5ba53819d1203: Status 404 returned error can't find the container with id b787d7587d038ae35b5eab6546ec01b5b1752182c3e76c7655f5ba53819d1203
Jul 30 10:01:27.969994 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:27.969964    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 10:01:28.231645 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:28.231612    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6" event={"ID":"aff02424-d441-4ba2-9304-0745ff5972b0","Type":"ContainerStarted","Data":"b787d7587d038ae35b5eab6546ec01b5b1752182c3e76c7655f5ba53819d1203"}
Jul 30 10:01:30.096977 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:30.096937    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:01:44.303870 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:44.303786    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6" event={"ID":"aff02424-d441-4ba2-9304-0745ff5972b0","Type":"ContainerStarted","Data":"91c8e1c53f8ca1547d0d7e6c040aad6d433e63746f23be803d9f7a3b8cd8c838"}
Jul 30 10:01:44.327477 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:44.325834    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6" podStartSLOduration=1.328928818 podStartE2EDuration="17.325810826s" totalImagesPullingTime="15.996882008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:01:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:01:44.321673426 +0000 UTC m=+740.815868932" watchObservedRunningTime="2026-07-30 10:01:44.325810826 +0000 UTC m=+740.820006333"
Jul 30 10:01:44.628217 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:44.628183    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:01:47.826961 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:47.826907    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:47.826961 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:47.826978    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:47.839842 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:47.839813    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:01:48.333220 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:01:48.333185    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-swzr6"
Jul 30 10:03:18.004068 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:03:18.004031    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:03:27.498627 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:03:27.498590    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:03:38.492819 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:03:38.492778    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:04:23.915950 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:04:23.915877    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:04:23.915950 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:04:23.915876    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:04:23.922132 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:04:23.922102    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:04:23.922414 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:04:23.922397    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:05:18.598362 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:05:18.598293    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:05:30.591631 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:05:30.591590    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:06:27.694851 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:06:27.694814    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:06:36.899090 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:06:36.899047    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:07:03.097793 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:07:03.097758    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:07:12.087239 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:07:12.087200    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:07:22.700574 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:07:22.700489    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:07:34.493795 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:07:34.493753    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:07:45.994352 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:07:45.994291    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:07:55.997453 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:07:55.997414    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:08:18.007648 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:08:18.007608    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:08:28.792937 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:08:28.792897    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:08:52.010783 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:08:52.010700    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:09:14.895666 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:09:14.895633    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:09:23.943195 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:09:23.943164    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:09:23.943737 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:09:23.943697    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:09:23.949056 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:09:23.949030    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:09:23.949426 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:09:23.949409    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:09:29.798718 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:09:29.798685    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:09:50.996581 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:09:50.996541    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:10:07.101520 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:10:07.101486    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:10:16.702244 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:10:16.702168    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:10:31.101053 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:10:31.101016    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:10:45.105129 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:10:45.105090    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:11:06.405877 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:11:06.405830    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:11:15.797115 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:11:15.797079    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:11:26.096457 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:11:26.096417    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:13:38.296641 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:13:38.296608    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:13:45.607637 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:13:45.607604    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:14:08.416613 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:14:08.416577    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:14:23.972201 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:14:23.972169    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:14:23.973250 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:14:23.973224    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:14:23.978013 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:14:23.977989    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:14:23.978960 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:14:23.978941    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:14:48.697932 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:14:48.697832    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:14:57.795114 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:14:57.795072    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:15:07.401734 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:15:07.401698    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:15:17.595267 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:15:17.595232    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:15:29.402094 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:15:29.402061    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:15:41.314021 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:15:41.313982    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:15:53.198122 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:15:53.198085    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:16:04.492581 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:04.492544    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:16:12.307355 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:12.307305    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:16:31.497856 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:31.497773    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:16:41.101184 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:41.101143    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:16:56.692739 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:56.692705    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:16:59.206617 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.206582    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"]
Jul 30 10:16:59.210394 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.210367    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.212743 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.212724    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-18824086-kserve-self-signed-certs\""
Jul 30 10:16:59.217086 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.216808    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"]
Jul 30 10:16:59.301313 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.301277    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d6p6\" (UniqueName: \"kubernetes.io/projected/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-kube-api-access-9d6p6\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.301526 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.301322    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-home\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.301526 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.301417    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-dshm\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.301526 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.301507    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-model-cache\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.301683 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.301544    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-tmp-dir\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.301683 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.301583    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-tls-certs\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.402711 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.402679    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-model-cache\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.402711 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.402716    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-tmp-dir\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.402922 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.402741    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-tls-certs\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.402922 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.402787    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d6p6\" (UniqueName: \"kubernetes.io/projected/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-kube-api-access-9d6p6\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.402922 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.402818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-home\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.403074 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.402984    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-dshm\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.403137 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.403104    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-model-cache\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.403215 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.403194    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-tmp-dir\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.403263 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.403227    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-home\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.405092 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.405060    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-dshm\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.405277 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.405262    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-tls-certs\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.411580 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.411557    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d6p6\" (UniqueName: \"kubernetes.io/projected/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-kube-api-access-9d6p6\") pod \"e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") " pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.519821 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.519740    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:16:59.653583 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:16:59.653557    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"]
Jul 30 10:16:59.656775 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:16:59.656741    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcdcf8ba7_35b2_4122_9d72_77ad5b6b2703.slice/crio-bb51c4011052124feaa3813a57f5285b4ff2c03250a72fb313f0369fc972267b WatchSource:0}: Error finding container bb51c4011052124feaa3813a57f5285b4ff2c03250a72fb313f0369fc972267b: Status 404 returned error can't find the container with id bb51c4011052124feaa3813a57f5285b4ff2c03250a72fb313f0369fc972267b
Jul 30 10:17:00.370237 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:00.370201    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98" event={"ID":"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703","Type":"ContainerStarted","Data":"d885dc1959f0c5cc230ee2a4172a54ac5e36e70e4533ca00af3379c6af4ef76d"}
Jul 30 10:17:00.370237 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:00.370243    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98" event={"ID":"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703","Type":"ContainerStarted","Data":"bb51c4011052124feaa3813a57f5285b4ff2c03250a72fb313f0369fc972267b"}
Jul 30 10:17:00.385811 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:00.385767    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98" podStartSLOduration=1.385751072 podStartE2EDuration="1.385751072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:16:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:17:00.384780267 +0000 UTC m=+1656.878975772" watchObservedRunningTime="2026-07-30 10:17:00.385751072 +0000 UTC m=+1656.879946577"
Jul 30 10:17:09.520555 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:09.520512    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:17:09.520555 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:09.520561    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:17:09.533060 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:09.533034    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:17:09.593987 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:09.593951    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:17:10.431588 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:10.431553    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:17:27.627086 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:27.627048    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"]
Jul 30 10:17:28.894106 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:28.894070    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:17:42.646170 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.646124    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98" podUID="cdcf8ba7-35b2-4122-9d72-77ad5b6b2703" containerName="main" containerID="cri-o://d885dc1959f0c5cc230ee2a4172a54ac5e36e70e4533ca00af3379c6af4ef76d" gracePeriod=45
Jul 30 10:17:42.898519 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.898453    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:17:42.981096 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.981066    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-model-cache\") pod \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") "
Jul 30 10:17:42.981096 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.981101    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-tls-certs\") pod \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") "
Jul 30 10:17:42.981389 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.981122    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-home\") pod \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") "
Jul 30 10:17:42.981389 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.981163    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-tmp-dir\") pod \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") "
Jul 30 10:17:42.981389 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.981195    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d6p6\" (UniqueName: \"kubernetes.io/projected/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-kube-api-access-9d6p6\") pod \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") "
Jul 30 10:17:42.981389 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.981231    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-dshm\") pod \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\" (UID: \"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703\") "
Jul 30 10:17:42.981389 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.981374    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-model-cache" (OuterVolumeSpecName: "model-cache") pod "cdcf8ba7-35b2-4122-9d72-77ad5b6b2703" (UID: "cdcf8ba7-35b2-4122-9d72-77ad5b6b2703"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:17:42.981650 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.981481    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-home" (OuterVolumeSpecName: "home") pod "cdcf8ba7-35b2-4122-9d72-77ad5b6b2703" (UID: "cdcf8ba7-35b2-4122-9d72-77ad5b6b2703"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:17:42.981650 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.981527    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cdcf8ba7-35b2-4122-9d72-77ad5b6b2703" (UID: "cdcf8ba7-35b2-4122-9d72-77ad5b6b2703"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:17:42.981650 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.981636    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:17:42.981763 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.981669    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:17:42.981763 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.981680    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:17:42.983424 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.983399    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-dshm" (OuterVolumeSpecName: "dshm") pod "cdcf8ba7-35b2-4122-9d72-77ad5b6b2703" (UID: "cdcf8ba7-35b2-4122-9d72-77ad5b6b2703"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:17:42.983672 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.983651    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-kube-api-access-9d6p6" (OuterVolumeSpecName: "kube-api-access-9d6p6") pod "cdcf8ba7-35b2-4122-9d72-77ad5b6b2703" (UID: "cdcf8ba7-35b2-4122-9d72-77ad5b6b2703"). InnerVolumeSpecName "kube-api-access-9d6p6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:17:42.983721 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:42.983691    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cdcf8ba7-35b2-4122-9d72-77ad5b6b2703" (UID: "cdcf8ba7-35b2-4122-9d72-77ad5b6b2703"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 10:17:43.082679 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:43.082642    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d6p6\" (UniqueName: \"kubernetes.io/projected/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-kube-api-access-9d6p6\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:17:43.082679 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:43.082673    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:17:43.082679 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:43.082683    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:17:43.293153 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:43.293069    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:17:43.568555 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:43.568471    2567 generic.go:356] "Generic (PLEG): container finished" podID="cdcf8ba7-35b2-4122-9d72-77ad5b6b2703" containerID="d885dc1959f0c5cc230ee2a4172a54ac5e36e70e4533ca00af3379c6af4ef76d" exitCode=0
Jul 30 10:17:43.568555 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:43.568513    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"
Jul 30 10:17:43.568728 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:43.568510    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98" event={"ID":"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703","Type":"ContainerDied","Data":"d885dc1959f0c5cc230ee2a4172a54ac5e36e70e4533ca00af3379c6af4ef76d"}
Jul 30 10:17:43.568728 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:43.568638    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98" event={"ID":"cdcf8ba7-35b2-4122-9d72-77ad5b6b2703","Type":"ContainerDied","Data":"bb51c4011052124feaa3813a57f5285b4ff2c03250a72fb313f0369fc972267b"}
Jul 30 10:17:43.568728 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:43.568662    2567 scope.go:117] "RemoveContainer" containerID="d885dc1959f0c5cc230ee2a4172a54ac5e36e70e4533ca00af3379c6af4ef76d"
Jul 30 10:17:43.577707 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:43.577687    2567 scope.go:117] "RemoveContainer" containerID="d885dc1959f0c5cc230ee2a4172a54ac5e36e70e4533ca00af3379c6af4ef76d"
Jul 30 10:17:43.577942 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:17:43.577924    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d885dc1959f0c5cc230ee2a4172a54ac5e36e70e4533ca00af3379c6af4ef76d\": container with ID starting with d885dc1959f0c5cc230ee2a4172a54ac5e36e70e4533ca00af3379c6af4ef76d not found: ID does not exist" containerID="d885dc1959f0c5cc230ee2a4172a54ac5e36e70e4533ca00af3379c6af4ef76d"
Jul 30 10:17:43.577988 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:43.577950    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d885dc1959f0c5cc230ee2a4172a54ac5e36e70e4533ca00af3379c6af4ef76d"} err="failed to get container status \"d885dc1959f0c5cc230ee2a4172a54ac5e36e70e4533ca00af3379c6af4ef76d\": rpc error: code = NotFound desc = could not find container \"d885dc1959f0c5cc230ee2a4172a54ac5e36e70e4533ca00af3379c6af4ef76d\": container with ID starting with d885dc1959f0c5cc230ee2a4172a54ac5e36e70e4533ca00af3379c6af4ef76d not found: ID does not exist"
Jul 30 10:17:43.592196 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:43.592168    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"]
Jul 30 10:17:43.595932 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:43.595910    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-18824086-kserve-694d6d7bd7-vsg98"]
Jul 30 10:17:44.063776 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:44.063746    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdcf8ba7-35b2-4122-9d72-77ad5b6b2703" path="/var/lib/kubelet/pods/cdcf8ba7-35b2-4122-9d72-77ad5b6b2703/volumes"
Jul 30 10:17:51.101430 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:17:51.101348    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:18:03.791389 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:18:03.791351    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:18:19.391545 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:18:19.391507    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:18:30.492308 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:18:30.492265    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:18:40.745403 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:18:40.745361    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:18:49.995605 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:18:49.995571    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:18:59.495895 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:18:59.495857    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:19:13.401512 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:19:13.401479    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:19:23.231581 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:19:23.231546    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:19:23.998296 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:19:23.998268    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:19:24.000437 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:19:24.000409    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:19:24.004093 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:19:24.004073    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:19:24.006180 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:19:24.006162    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:19:37.791896 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:19:37.791862    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:19:47.200361 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:19:47.200300    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:19:59.397055 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:19:59.397022    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:20:11.390431 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:20:11.390385    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:20:25.793318 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:20:25.793284    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:20:34.794757 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:20:34.794723    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:20:53.599980 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:20:53.599901    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:21:19.293694 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:21:19.293659    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:21:28.793563 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:21:28.793529    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:21:42.389439 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:21:42.389402    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:21:56.390830 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:21:56.390794    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:22:05.701389 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:22:05.701354    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:22:15.793535 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:22:15.793499    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:22:26.892253 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:22:26.892216    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:22:55.198638 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:22:55.198604    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:23:06.690701 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:23:06.690665    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:23:29.691881 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:23:29.691844    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:23:41.387765 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:23:41.387729    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:24:04.831554 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:24:04.831458    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:24:15.188315 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:24:15.188278    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:24:24.028608 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:24:24.028573    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:24:24.030448 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:24:24.030418    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:24:24.034237 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:24:24.034215    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:24:24.036424 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:24:24.036404    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:24:40.792819 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:24:40.792780    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:25:01.791790 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:25:01.791754    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:26:32.994149 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:26:32.994114    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:26:44.992266 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:26:44.992230    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:26:55.591359 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:26:55.591258    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:27:34.292329 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:27:34.292292    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:28:11.098502 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:28:11.098466    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:29:24.054757 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:29:24.054730    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:29:24.057055 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:29:24.057026    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:29:24.061414 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:29:24.061388    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:29:24.063811 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:29:24.063787    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:29:57.399687 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:29:57.399654    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:30:12.011063 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:12.011022    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:30:47.502191 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.502153    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"]
Jul 30 10:30:47.502736 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.502720    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdcf8ba7-35b2-4122-9d72-77ad5b6b2703" containerName="main"
Jul 30 10:30:47.502809 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.502740    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdcf8ba7-35b2-4122-9d72-77ad5b6b2703" containerName="main"
Jul 30 10:30:47.502857 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.502846    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdcf8ba7-35b2-4122-9d72-77ad5b6b2703" containerName="main"
Jul 30 10:30:47.505953 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.505934    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.508406 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.508381    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2c013e89\"/\"kube-root-ca.crt\""
Jul 30 10:30:47.509318 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.509295    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2c013e89\"/\"openshift-service-ca.crt\""
Jul 30 10:30:47.509442 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.509298    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2c013e89\"/\"default-dockercfg-w84pw\""
Jul 30 10:30:47.509442 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.509433    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2c013e89\"/\"e2e-lifecycle-model-2c013e89-kserve-self-signed-certs\""
Jul 30 10:30:47.514090 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.514071    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"]
Jul 30 10:30:47.581231 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.581190    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-tmp-dir\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.581231 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.581230    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/475d0d8f-5c61-467b-8c5c-88e98fcf270b-tls-certs\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.581470 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.581270    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-model-cache\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.581470 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.581303    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-dshm\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.581470 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.581360    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-home\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.581470 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.581382    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvgr6\" (UniqueName: \"kubernetes.io/projected/475d0d8f-5c61-467b-8c5c-88e98fcf270b-kube-api-access-vvgr6\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.682234 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.682196    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-model-cache\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.682234 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.682236    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-dshm\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.682520 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.682275    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-home\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.682520 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.682299    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvgr6\" (UniqueName: \"kubernetes.io/projected/475d0d8f-5c61-467b-8c5c-88e98fcf270b-kube-api-access-vvgr6\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.682520 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.682355    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-tmp-dir\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.682520 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.682500    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/475d0d8f-5c61-467b-8c5c-88e98fcf270b-tls-certs\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.682746 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.682726    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-tmp-dir\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.682807 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.682790    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-model-cache\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.682861 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.682837    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-home\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.684704 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.684679    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-dshm\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.685132 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.685112    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/475d0d8f-5c61-467b-8c5c-88e98fcf270b-tls-certs\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.692016 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.691982    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvgr6\" (UniqueName: \"kubernetes.io/projected/475d0d8f-5c61-467b-8c5c-88e98fcf270b-kube-api-access-vvgr6\") pod \"e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") " pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.822843 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.822747    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:47.958947 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:47.958921    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"]
Jul 30 10:30:47.961097 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:30:47.961071    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod475d0d8f_5c61_467b_8c5c_88e98fcf270b.slice/crio-f0c0c1cea15d38f530d7d24691cf67a052bd1d46c7340146b640c3755c1209db WatchSource:0}: Error finding container f0c0c1cea15d38f530d7d24691cf67a052bd1d46c7340146b640c3755c1209db: Status 404 returned error can't find the container with id f0c0c1cea15d38f530d7d24691cf67a052bd1d46c7340146b640c3755c1209db
Jul 30 10:30:48.051820 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:48.051772    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598" event={"ID":"475d0d8f-5c61-467b-8c5c-88e98fcf270b","Type":"ContainerStarted","Data":"c2f3e93e778cbb372526c06a921d94194f7c4629c2ee22b285139bd28955103f"}
Jul 30 10:30:48.052012 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:48.051839    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598" event={"ID":"475d0d8f-5c61-467b-8c5c-88e98fcf270b","Type":"ContainerStarted","Data":"f0c0c1cea15d38f530d7d24691cf67a052bd1d46c7340146b640c3755c1209db"}
Jul 30 10:30:48.068675 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:48.068622    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598" podStartSLOduration=1.068606793 podStartE2EDuration="1.068606793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:30:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:30:48.068139171 +0000 UTC m=+2484.562334679" watchObservedRunningTime="2026-07-30 10:30:48.068606793 +0000 UTC m=+2484.562802277"
Jul 30 10:30:50.887851 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:50.887812    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:30:57.823635 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:57.823602    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:57.824162 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:57.823654    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:57.836062 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:57.836031    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:30:58.107352 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:30:58.107251    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:31:17.492767 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:31:17.492670    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:31:35.897325 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:31:35.897290    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:31:44.793688 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:31:44.793649    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:31:46.231942 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:31:46.231908    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"]
Jul 30 10:32:01.251109 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.251050    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598" podUID="475d0d8f-5c61-467b-8c5c-88e98fcf270b" containerName="main" containerID="cri-o://c2f3e93e778cbb372526c06a921d94194f7c4629c2ee22b285139bd28955103f" gracePeriod=45
Jul 30 10:32:01.376384 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.376326    2567 generic.go:356] "Generic (PLEG): container finished" podID="475d0d8f-5c61-467b-8c5c-88e98fcf270b" containerID="c2f3e93e778cbb372526c06a921d94194f7c4629c2ee22b285139bd28955103f" exitCode=0
Jul 30 10:32:01.376384 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.376367    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598" event={"ID":"475d0d8f-5c61-467b-8c5c-88e98fcf270b","Type":"ContainerDied","Data":"c2f3e93e778cbb372526c06a921d94194f7c4629c2ee22b285139bd28955103f"}
Jul 30 10:32:01.508870 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.508812    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:32:01.643876 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.643827    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-model-cache\") pod \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") "
Jul 30 10:32:01.644077 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.643892    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvgr6\" (UniqueName: \"kubernetes.io/projected/475d0d8f-5c61-467b-8c5c-88e98fcf270b-kube-api-access-vvgr6\") pod \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") "
Jul 30 10:32:01.644077 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.643928    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-tmp-dir\") pod \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") "
Jul 30 10:32:01.644077 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.643963    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-dshm\") pod \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") "
Jul 30 10:32:01.644077 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.644059    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-home\") pod \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") "
Jul 30 10:32:01.644295 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.644165    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/475d0d8f-5c61-467b-8c5c-88e98fcf270b-tls-certs\") pod \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\" (UID: \"475d0d8f-5c61-467b-8c5c-88e98fcf270b\") "
Jul 30 10:32:01.644295 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.644168    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-model-cache" (OuterVolumeSpecName: "model-cache") pod "475d0d8f-5c61-467b-8c5c-88e98fcf270b" (UID: "475d0d8f-5c61-467b-8c5c-88e98fcf270b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:32:01.644295 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.644275    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "475d0d8f-5c61-467b-8c5c-88e98fcf270b" (UID: "475d0d8f-5c61-467b-8c5c-88e98fcf270b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:32:01.644458 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.644329    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-home" (OuterVolumeSpecName: "home") pod "475d0d8f-5c61-467b-8c5c-88e98fcf270b" (UID: "475d0d8f-5c61-467b-8c5c-88e98fcf270b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:32:01.644494 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.644463    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:32:01.644494 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.644476    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:32:01.644494 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.644484    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:32:01.646346 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.646318    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-dshm" (OuterVolumeSpecName: "dshm") pod "475d0d8f-5c61-467b-8c5c-88e98fcf270b" (UID: "475d0d8f-5c61-467b-8c5c-88e98fcf270b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:32:01.646508 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.646486    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/475d0d8f-5c61-467b-8c5c-88e98fcf270b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "475d0d8f-5c61-467b-8c5c-88e98fcf270b" (UID: "475d0d8f-5c61-467b-8c5c-88e98fcf270b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 10:32:01.646580 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.646528    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/475d0d8f-5c61-467b-8c5c-88e98fcf270b-kube-api-access-vvgr6" (OuterVolumeSpecName: "kube-api-access-vvgr6") pod "475d0d8f-5c61-467b-8c5c-88e98fcf270b" (UID: "475d0d8f-5c61-467b-8c5c-88e98fcf270b"). InnerVolumeSpecName "kube-api-access-vvgr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:32:01.745347 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.745301    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/475d0d8f-5c61-467b-8c5c-88e98fcf270b-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:32:01.745513 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.745331    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvgr6\" (UniqueName: \"kubernetes.io/projected/475d0d8f-5c61-467b-8c5c-88e98fcf270b-kube-api-access-vvgr6\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:32:01.745513 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:01.745375    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/475d0d8f-5c61-467b-8c5c-88e98fcf270b-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:32:02.382048 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:02.382014    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598" event={"ID":"475d0d8f-5c61-467b-8c5c-88e98fcf270b","Type":"ContainerDied","Data":"f0c0c1cea15d38f530d7d24691cf67a052bd1d46c7340146b640c3755c1209db"}
Jul 30 10:32:02.382048 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:02.382049    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"
Jul 30 10:32:02.382535 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:02.382066    2567 scope.go:117] "RemoveContainer" containerID="c2f3e93e778cbb372526c06a921d94194f7c4629c2ee22b285139bd28955103f"
Jul 30 10:32:02.397433 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:02.397407    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"]
Jul 30 10:32:02.401233 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:02.401210    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-2c013e89/e2e-lifecycle-model-2c013e89-kserve-6cd5fbcc5f-47598"]
Jul 30 10:32:04.062502 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:04.062470    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="475d0d8f-5c61-467b-8c5c-88e98fcf270b" path="/var/lib/kubelet/pods/475d0d8f-5c61-467b-8c5c-88e98fcf270b/volumes"
Jul 30 10:32:38.002500 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:32:38.002441    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:33:01.992718 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:33:01.992680    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:33:11.591080 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:33:11.591036    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:33:59.700280 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:33:59.700245    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:34:24.081807 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:24.081726    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:34:24.085688 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:24.085661    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:34:24.087840 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:24.087821    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:34:24.092428 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:24.092392    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:34:29.603318 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:29.603282    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:34:52.707355 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.707310    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"]
Jul 30 10:34:52.707790 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.707658    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="475d0d8f-5c61-467b-8c5c-88e98fcf270b" containerName="main"
Jul 30 10:34:52.707790 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.707669    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="475d0d8f-5c61-467b-8c5c-88e98fcf270b" containerName="main"
Jul 30 10:34:52.707790 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.707739    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="475d0d8f-5c61-467b-8c5c-88e98fcf270b" containerName="main"
Jul 30 10:34:52.710765 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.710747    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.714589 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.714563    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-98c9c7\"/\"test-model-98c9c7-kserve-self-signed-certs\""
Jul 30 10:34:52.714705 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.714563    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-98c9c7\"/\"kube-root-ca.crt\""
Jul 30 10:34:52.714705 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.714604    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-98c9c7\"/\"default-dockercfg-l4mvd\""
Jul 30 10:34:52.714705 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.714572    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-98c9c7\"/\"openshift-service-ca.crt\""
Jul 30 10:34:52.721982 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.721961    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"]
Jul 30 10:34:52.763856 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.763824    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xqzv\" (UniqueName: \"kubernetes.io/projected/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-kube-api-access-8xqzv\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.764009 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.763869    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-tmp-dir\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.764009 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.763947    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-home\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.764009 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.764000    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-dshm\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.764165 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.764030    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-model-cache\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.764165 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.764077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-tls-certs\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.865278 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.865243    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xqzv\" (UniqueName: \"kubernetes.io/projected/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-kube-api-access-8xqzv\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.865278 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.865281    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-tmp-dir\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.865526 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.865311    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-home\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.865526 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.865356    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-dshm\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.865526 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.865383    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-model-cache\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.865526 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.865400    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-tls-certs\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.865743 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.865708    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-tmp-dir\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.865865 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.865836    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-home\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.865865 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.865856    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-model-cache\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.867767 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.867748    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-dshm\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.867914 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.867899    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-tls-certs\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:52.873175 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:52.873145    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xqzv\" (UniqueName: \"kubernetes.io/projected/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-kube-api-access-8xqzv\") pod \"test-model-98c9c7-kserve-66cf6cd467-46mqb\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") " pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:53.020153 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:53.020053    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:34:53.150100 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:53.150069    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"]
Jul 30 10:34:53.153372 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:34:53.153325    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34a56e53_1f4f_4ee8_9be7_00fb4084fb1b.slice/crio-ff67fafecd6a934b9b3e80782c8ed392e0157a2638991537c46cec49b47da883 WatchSource:0}: Error finding container ff67fafecd6a934b9b3e80782c8ed392e0157a2638991537c46cec49b47da883: Status 404 returned error can't find the container with id ff67fafecd6a934b9b3e80782c8ed392e0157a2638991537c46cec49b47da883
Jul 30 10:34:54.136508 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:54.136471    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb" event={"ID":"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b","Type":"ContainerStarted","Data":"882e8603d1d3c01cb3a95794594af66673146d4ad0d2976ed9063685fef4b2c0"}
Jul 30 10:34:54.136508 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:54.136512    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb" event={"ID":"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b","Type":"ContainerStarted","Data":"ff67fafecd6a934b9b3e80782c8ed392e0157a2638991537c46cec49b47da883"}
Jul 30 10:34:54.153372 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:54.153308    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb" podStartSLOduration=2.153293425 podStartE2EDuration="2.153293425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:34:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:34:54.152469209 +0000 UTC m=+2730.646664712" watchObservedRunningTime="2026-07-30 10:34:54.153293425 +0000 UTC m=+2730.647488930"
Jul 30 10:34:57.190798 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:34:57.190721    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:35:03.020656 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:03.020616    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:35:03.020656 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:03.020664    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:35:03.033076 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:03.033049    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:35:03.188782 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:03.188751    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:35:15.014957 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.014923    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"]
Jul 30 10:35:15.018291 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.018270    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.021934 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.021904    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-fde6bc\"/\"openshift-service-ca.crt\""
Jul 30 10:35:15.021934 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.021934    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-fde6bc\"/\"kube-root-ca.crt\""
Jul 30 10:35:15.022123 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.021912    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-fde6bc\"/\"default-dockercfg-hzrsp\""
Jul 30 10:35:15.022123 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.021912    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-fde6bc\"/\"test-model-fde6bc-kserve-self-signed-certs\""
Jul 30 10:35:15.028299 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.028272    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"]
Jul 30 10:35:15.167272 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.167233    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad4ae85-671d-4e98-8c7f-aa325f8406c8-tls-certs\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.167470 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.167279    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl2zk\" (UniqueName: \"kubernetes.io/projected/bad4ae85-671d-4e98-8c7f-aa325f8406c8-kube-api-access-jl2zk\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.167470 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.167380    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-dshm\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.167470 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.167399    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-home\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.167580 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.167463    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-model-cache\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.167580 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.167517    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-tmp-dir\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.268200 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.268105    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-dshm\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.268200 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.268141    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-home\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.268200 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.268164    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-model-cache\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.268546 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.268205    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-tmp-dir\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.268546 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.268234    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad4ae85-671d-4e98-8c7f-aa325f8406c8-tls-certs\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.268546 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.268261    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl2zk\" (UniqueName: \"kubernetes.io/projected/bad4ae85-671d-4e98-8c7f-aa325f8406c8-kube-api-access-jl2zk\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.268701 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.268636    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-tmp-dir\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.268701 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.268689    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-home\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.268794 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.268775    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-model-cache\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.270508 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.270488    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-dshm\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.270683 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.270668    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad4ae85-671d-4e98-8c7f-aa325f8406c8-tls-certs\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.277939 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.277917    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl2zk\" (UniqueName: \"kubernetes.io/projected/bad4ae85-671d-4e98-8c7f-aa325f8406c8-kube-api-access-jl2zk\") pod \"test-model-fde6bc-kserve-7b6765f7cc-7jb5b\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") " pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.328502 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.328462    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:15.467215 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:15.467182    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"]
Jul 30 10:35:15.469748 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:35:15.469721    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbad4ae85_671d_4e98_8c7f_aa325f8406c8.slice/crio-14e13ab05d98256113c2d786321fcefd2ab5386888a06ace47fa86298067ba53 WatchSource:0}: Error finding container 14e13ab05d98256113c2d786321fcefd2ab5386888a06ace47fa86298067ba53: Status 404 returned error can't find the container with id 14e13ab05d98256113c2d786321fcefd2ab5386888a06ace47fa86298067ba53
Jul 30 10:35:16.237097 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:16.237061    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b" event={"ID":"bad4ae85-671d-4e98-8c7f-aa325f8406c8","Type":"ContainerStarted","Data":"c3100b755c4fd2aa85762babee4b42b8dd17e1da0ef05ae0489d72ba05747396"}
Jul 30 10:35:16.237097 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:16.237103    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b" event={"ID":"bad4ae85-671d-4e98-8c7f-aa325f8406c8","Type":"ContainerStarted","Data":"14e13ab05d98256113c2d786321fcefd2ab5386888a06ace47fa86298067ba53"}
Jul 30 10:35:16.255757 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:16.255701    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b" podStartSLOduration=2.255685039 podStartE2EDuration="2.255685039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:35:16.254079591 +0000 UTC m=+2752.748275098" watchObservedRunningTime="2026-07-30 10:35:16.255685039 +0000 UTC m=+2752.749880544"
Jul 30 10:35:19.197776 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:19.197743    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:35:25.329259 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:25.329228    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:25.329682 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:25.329283    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:25.341684 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:25.341661    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:26.294263 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:26.294227    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:35:31.100795 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:35:31.100759    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:36:07.332472 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:07.332441    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"]
Jul 30 10:36:08.246958 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:08.246924    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"]
Jul 30 10:36:12.801928 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:12.801889    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:36:22.351540 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.351495    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb" podUID="34a56e53-1f4f-4ee8-9be7-00fb4084fb1b" containerName="main" containerID="cri-o://882e8603d1d3c01cb3a95794594af66673146d4ad0d2976ed9063685fef4b2c0" gracePeriod=45
Jul 30 10:36:22.537898 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.537869    2567 generic.go:356] "Generic (PLEG): container finished" podID="34a56e53-1f4f-4ee8-9be7-00fb4084fb1b" containerID="882e8603d1d3c01cb3a95794594af66673146d4ad0d2976ed9063685fef4b2c0" exitCode=0
Jul 30 10:36:22.538047 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.537939    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb" event={"ID":"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b","Type":"ContainerDied","Data":"882e8603d1d3c01cb3a95794594af66673146d4ad0d2976ed9063685fef4b2c0"}
Jul 30 10:36:22.615378 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.615315    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:36:22.735647 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.735613    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-tls-certs\") pod \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") "
Jul 30 10:36:22.735824 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.735678    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-tmp-dir\") pod \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") "
Jul 30 10:36:22.735824 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.735699    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-dshm\") pod \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") "
Jul 30 10:36:22.735824 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.735738    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xqzv\" (UniqueName: \"kubernetes.io/projected/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-kube-api-access-8xqzv\") pod \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") "
Jul 30 10:36:22.735824 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.735787    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-model-cache\") pod \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") "
Jul 30 10:36:22.735824 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.735817    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-home\") pod \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\" (UID: \"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b\") "
Jul 30 10:36:22.736078 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.735994    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "34a56e53-1f4f-4ee8-9be7-00fb4084fb1b" (UID: "34a56e53-1f4f-4ee8-9be7-00fb4084fb1b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:36:22.736078 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.736064    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:36:22.736179 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.736145    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-home" (OuterVolumeSpecName: "home") pod "34a56e53-1f4f-4ee8-9be7-00fb4084fb1b" (UID: "34a56e53-1f4f-4ee8-9be7-00fb4084fb1b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:36:22.736233 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.736164    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-model-cache" (OuterVolumeSpecName: "model-cache") pod "34a56e53-1f4f-4ee8-9be7-00fb4084fb1b" (UID: "34a56e53-1f4f-4ee8-9be7-00fb4084fb1b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:36:22.737951 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.737924    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "34a56e53-1f4f-4ee8-9be7-00fb4084fb1b" (UID: "34a56e53-1f4f-4ee8-9be7-00fb4084fb1b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 10:36:22.737951 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.737945    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-dshm" (OuterVolumeSpecName: "dshm") pod "34a56e53-1f4f-4ee8-9be7-00fb4084fb1b" (UID: "34a56e53-1f4f-4ee8-9be7-00fb4084fb1b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:36:22.738090 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.737989    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-kube-api-access-8xqzv" (OuterVolumeSpecName: "kube-api-access-8xqzv") pod "34a56e53-1f4f-4ee8-9be7-00fb4084fb1b" (UID: "34a56e53-1f4f-4ee8-9be7-00fb4084fb1b"). InnerVolumeSpecName "kube-api-access-8xqzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:36:22.836816 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.836766    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xqzv\" (UniqueName: \"kubernetes.io/projected/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-kube-api-access-8xqzv\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:36:22.836816 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.836808    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:36:22.836816 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.836820    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:36:22.836816 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.836828    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:36:22.836816 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:22.836835    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:36:23.266262 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.266197    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b" podUID="bad4ae85-671d-4e98-8c7f-aa325f8406c8" containerName="main" containerID="cri-o://c3100b755c4fd2aa85762babee4b42b8dd17e1da0ef05ae0489d72ba05747396" gracePeriod=45
Jul 30 10:36:23.518318 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.518262    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:36:23.543641 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.543605    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad4ae85-671d-4e98-8c7f-aa325f8406c8-tls-certs\") pod \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") "
Jul 30 10:36:23.543806 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.543667    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-dshm\") pod \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") "
Jul 30 10:36:23.543806 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.543729    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-model-cache\") pod \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") "
Jul 30 10:36:23.543806 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.543759    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-tmp-dir\") pod \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") "
Jul 30 10:36:23.543806 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.543798    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-home\") pod \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") "
Jul 30 10:36:23.544009 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.543869    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jl2zk\" (UniqueName: \"kubernetes.io/projected/bad4ae85-671d-4e98-8c7f-aa325f8406c8-kube-api-access-jl2zk\") pod \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\" (UID: \"bad4ae85-671d-4e98-8c7f-aa325f8406c8\") "
Jul 30 10:36:23.544938 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.544901    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-model-cache" (OuterVolumeSpecName: "model-cache") pod "bad4ae85-671d-4e98-8c7f-aa325f8406c8" (UID: "bad4ae85-671d-4e98-8c7f-aa325f8406c8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:36:23.545747 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.545534    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bad4ae85-671d-4e98-8c7f-aa325f8406c8" (UID: "bad4ae85-671d-4e98-8c7f-aa325f8406c8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:36:23.546014 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.545990    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-home" (OuterVolumeSpecName: "home") pod "bad4ae85-671d-4e98-8c7f-aa325f8406c8" (UID: "bad4ae85-671d-4e98-8c7f-aa325f8406c8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:36:23.548187 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.548158    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-dshm" (OuterVolumeSpecName: "dshm") pod "bad4ae85-671d-4e98-8c7f-aa325f8406c8" (UID: "bad4ae85-671d-4e98-8c7f-aa325f8406c8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:36:23.549978 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.549896    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bad4ae85-671d-4e98-8c7f-aa325f8406c8-kube-api-access-jl2zk" (OuterVolumeSpecName: "kube-api-access-jl2zk") pod "bad4ae85-671d-4e98-8c7f-aa325f8406c8" (UID: "bad4ae85-671d-4e98-8c7f-aa325f8406c8"). InnerVolumeSpecName "kube-api-access-jl2zk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:36:23.550788 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.550612    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb" event={"ID":"34a56e53-1f4f-4ee8-9be7-00fb4084fb1b","Type":"ContainerDied","Data":"ff67fafecd6a934b9b3e80782c8ed392e0157a2638991537c46cec49b47da883"}
Jul 30 10:36:23.550788 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.550673    2567 scope.go:117] "RemoveContainer" containerID="882e8603d1d3c01cb3a95794594af66673146d4ad0d2976ed9063685fef4b2c0"
Jul 30 10:36:23.550788 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.550660    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"
Jul 30 10:36:23.551842 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.551777    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bad4ae85-671d-4e98-8c7f-aa325f8406c8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bad4ae85-671d-4e98-8c7f-aa325f8406c8" (UID: "bad4ae85-671d-4e98-8c7f-aa325f8406c8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 10:36:23.552202 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.552178    2567 generic.go:356] "Generic (PLEG): container finished" podID="bad4ae85-671d-4e98-8c7f-aa325f8406c8" containerID="c3100b755c4fd2aa85762babee4b42b8dd17e1da0ef05ae0489d72ba05747396" exitCode=0
Jul 30 10:36:23.552377 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.552217    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b" event={"ID":"bad4ae85-671d-4e98-8c7f-aa325f8406c8","Type":"ContainerDied","Data":"c3100b755c4fd2aa85762babee4b42b8dd17e1da0ef05ae0489d72ba05747396"}
Jul 30 10:36:23.552377 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.552245    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b" event={"ID":"bad4ae85-671d-4e98-8c7f-aa325f8406c8","Type":"ContainerDied","Data":"14e13ab05d98256113c2d786321fcefd2ab5386888a06ace47fa86298067ba53"}
Jul 30 10:36:23.552377 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.552248    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"
Jul 30 10:36:23.567296 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.567274    2567 scope.go:117] "RemoveContainer" containerID="c3100b755c4fd2aa85762babee4b42b8dd17e1da0ef05ae0489d72ba05747396"
Jul 30 10:36:23.575027 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.575004    2567 scope.go:117] "RemoveContainer" containerID="c3100b755c4fd2aa85762babee4b42b8dd17e1da0ef05ae0489d72ba05747396"
Jul 30 10:36:23.575274 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:36:23.575255    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3100b755c4fd2aa85762babee4b42b8dd17e1da0ef05ae0489d72ba05747396\": container with ID starting with c3100b755c4fd2aa85762babee4b42b8dd17e1da0ef05ae0489d72ba05747396 not found: ID does not exist" containerID="c3100b755c4fd2aa85762babee4b42b8dd17e1da0ef05ae0489d72ba05747396"
Jul 30 10:36:23.575355 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.575281    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3100b755c4fd2aa85762babee4b42b8dd17e1da0ef05ae0489d72ba05747396"} err="failed to get container status \"c3100b755c4fd2aa85762babee4b42b8dd17e1da0ef05ae0489d72ba05747396\": rpc error: code = NotFound desc = could not find container \"c3100b755c4fd2aa85762babee4b42b8dd17e1da0ef05ae0489d72ba05747396\": container with ID starting with c3100b755c4fd2aa85762babee4b42b8dd17e1da0ef05ae0489d72ba05747396 not found: ID does not exist"
Jul 30 10:36:23.579394 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.579373    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"]
Jul 30 10:36:23.583182 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.583163    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-fde6bc/test-model-fde6bc-kserve-7b6765f7cc-7jb5b"]
Jul 30 10:36:23.593172 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.593152    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"]
Jul 30 10:36:23.596980 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.596961    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-98c9c7/test-model-98c9c7-kserve-66cf6cd467-46mqb"]
Jul 30 10:36:23.645149 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.645118    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jl2zk\" (UniqueName: \"kubernetes.io/projected/bad4ae85-671d-4e98-8c7f-aa325f8406c8-kube-api-access-jl2zk\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:36:23.645149 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.645143    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad4ae85-671d-4e98-8c7f-aa325f8406c8-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:36:23.645149 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.645153    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:36:23.645372 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.645161    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:36:23.645372 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.645188    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:36:23.645372 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:23.645196    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bad4ae85-671d-4e98-8c7f-aa325f8406c8-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:36:24.063959 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:24.063931    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34a56e53-1f4f-4ee8-9be7-00fb4084fb1b" path="/var/lib/kubelet/pods/34a56e53-1f4f-4ee8-9be7-00fb4084fb1b/volumes"
Jul 30 10:36:24.064319 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:24.064307    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bad4ae85-671d-4e98-8c7f-aa325f8406c8" path="/var/lib/kubelet/pods/bad4ae85-671d-4e98-8c7f-aa325f8406c8/volumes"
Jul 30 10:36:27.895787 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:27.895747    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:36:38.729150 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:38.729117    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:36:53.501001 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:36:53.500963    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:37:24.500372 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:24.500259    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:37:44.727078 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:44.727042    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj"]
Jul 30 10:37:44.727485 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:44.727383    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34a56e53-1f4f-4ee8-9be7-00fb4084fb1b" containerName="main"
Jul 30 10:37:44.727485 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:44.727395    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="34a56e53-1f4f-4ee8-9be7-00fb4084fb1b" containerName="main"
Jul 30 10:37:44.727485 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:44.727403    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bad4ae85-671d-4e98-8c7f-aa325f8406c8" containerName="main"
Jul 30 10:37:44.727485 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:44.727408    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bad4ae85-671d-4e98-8c7f-aa325f8406c8" containerName="main"
Jul 30 10:37:44.727485 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:44.727481    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="34a56e53-1f4f-4ee8-9be7-00fb4084fb1b" containerName="main"
Jul 30 10:37:44.727653 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:44.727491    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bad4ae85-671d-4e98-8c7f-aa325f8406c8" containerName="main"
Jul 30 10:37:44.730536 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:44.730520    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj"
Jul 30 10:37:44.733539 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:44.733518    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j6dj6\""
Jul 30 10:37:44.733744 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:44.733728    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 10:37:44.740006 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:44.739983    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj"]
Jul 30 10:37:44.826957 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:44.826921    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13909b78-f5ad-4975-b140-b08cc830a6ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b89f96-nlswj\" (UID: \"13909b78-f5ad-4975-b140-b08cc830a6ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj"
Jul 30 10:37:44.927901 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:44.927864    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13909b78-f5ad-4975-b140-b08cc830a6ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b89f96-nlswj\" (UID: \"13909b78-f5ad-4975-b140-b08cc830a6ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj"
Jul 30 10:37:44.928541 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:44.928523    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13909b78-f5ad-4975-b140-b08cc830a6ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b89f96-nlswj\" (UID: \"13909b78-f5ad-4975-b140-b08cc830a6ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj"
Jul 30 10:37:45.041356 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:45.041251    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj"
Jul 30 10:37:45.177027 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:45.176986    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj"]
Jul 30 10:37:45.179378 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:37:45.179328    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13909b78_f5ad_4975_b140_b08cc830a6ff.slice/crio-9b2d4075a6c95fdf2252c460659573079fbbcec0bd8fcc7ca22387a7ff015b28 WatchSource:0}: Error finding container 9b2d4075a6c95fdf2252c460659573079fbbcec0bd8fcc7ca22387a7ff015b28: Status 404 returned error can't find the container with id 9b2d4075a6c95fdf2252c460659573079fbbcec0bd8fcc7ca22387a7ff015b28
Jul 30 10:37:45.181235 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:45.181217    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 10:37:45.924405 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:45.924366    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj" event={"ID":"13909b78-f5ad-4975-b140-b08cc830a6ff","Type":"ContainerStarted","Data":"9b2d4075a6c95fdf2252c460659573079fbbcec0bd8fcc7ca22387a7ff015b28"}
Jul 30 10:37:47.937081 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:47.937045    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj" event={"ID":"13909b78-f5ad-4975-b140-b08cc830a6ff","Type":"ContainerStarted","Data":"d21e604f3dbed4667b024867a610633d5b693b1d00fec52e285fe330f614a627"}
Jul 30 10:37:47.955229 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:47.955122    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj" podStartSLOduration=1.434317468 podStartE2EDuration="3.955106847s" totalImagesPullingTime="2.520789379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:37:47.954813764 +0000 UTC m=+2904.449009271" watchObservedRunningTime="2026-07-30 10:37:47.955106847 +0000 UTC m=+2904.449302353"
Jul 30 10:37:48.943009 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:48.942971    2567 generic.go:356] "Generic (PLEG): container finished" podID="13909b78-f5ad-4975-b140-b08cc830a6ff" containerID="d21e604f3dbed4667b024867a610633d5b693b1d00fec52e285fe330f614a627" exitCode=0
Jul 30 10:37:48.943402 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:48.943061    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj" event={"ID":"13909b78-f5ad-4975-b140-b08cc830a6ff","Type":"ContainerDied","Data":"d21e604f3dbed4667b024867a610633d5b693b1d00fec52e285fe330f614a627"}
Jul 30 10:37:50.077358 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:50.077315    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj"
Jul 30 10:37:50.179424 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:50.179383    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13909b78-f5ad-4975-b140-b08cc830a6ff-maas-api-service-ca\") pod \"13909b78-f5ad-4975-b140-b08cc830a6ff\" (UID: \"13909b78-f5ad-4975-b140-b08cc830a6ff\") "
Jul 30 10:37:50.179800 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:50.179776    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13909b78-f5ad-4975-b140-b08cc830a6ff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "13909b78-f5ad-4975-b140-b08cc830a6ff" (UID: "13909b78-f5ad-4975-b140-b08cc830a6ff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 10:37:50.280994 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:50.280899    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/13909b78-f5ad-4975-b140-b08cc830a6ff-maas-api-service-ca\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:37:50.953415 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:50.953386    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj"
Jul 30 10:37:50.953627 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:50.953441    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj" event={"ID":"13909b78-f5ad-4975-b140-b08cc830a6ff","Type":"ContainerDied","Data":"9b2d4075a6c95fdf2252c460659573079fbbcec0bd8fcc7ca22387a7ff015b28"}
Jul 30 10:37:50.953627 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:37:50.953488    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b2d4075a6c95fdf2252c460659573079fbbcec0bd8fcc7ca22387a7ff015b28"
Jul 30 10:38:00.481207 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:00.481168    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj"]
Jul 30 10:38:00.483647 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:00.483622    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b89f96-nlswj"]
Jul 30 10:38:02.062877 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:02.062802    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13909b78-f5ad-4975-b140-b08cc830a6ff" path="/var/lib/kubelet/pods/13909b78-f5ad-4975-b140-b08cc830a6ff/volumes"
Jul 30 10:38:02.992405 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:02.992369    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:38:07.314081 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:07.314044    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw"]
Jul 30 10:38:07.314519 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:07.314404    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13909b78-f5ad-4975-b140-b08cc830a6ff" containerName="revoke-keys"
Jul 30 10:38:07.314519 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:07.314416    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="13909b78-f5ad-4975-b140-b08cc830a6ff" containerName="revoke-keys"
Jul 30 10:38:07.314519 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:07.314480    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="13909b78-f5ad-4975-b140-b08cc830a6ff" containerName="revoke-keys"
Jul 30 10:38:07.317374 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:07.317356    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw"
Jul 30 10:38:07.319993 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:07.319969    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 10:38:07.320129 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:07.320043    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j6dj6\""
Jul 30 10:38:07.321754 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:07.321730    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f42bfb2-0159-44cb-8538-6061d3904104-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-220161-6vjpw\" (UID: \"1f42bfb2-0159-44cb-8538-6061d3904104\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw"
Jul 30 10:38:07.325703 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:07.325680    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw"]
Jul 30 10:38:07.422378 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:07.422332    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f42bfb2-0159-44cb-8538-6061d3904104-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-220161-6vjpw\" (UID: \"1f42bfb2-0159-44cb-8538-6061d3904104\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw"
Jul 30 10:38:07.422938 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:07.422917    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f42bfb2-0159-44cb-8538-6061d3904104-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-220161-6vjpw\" (UID: \"1f42bfb2-0159-44cb-8538-6061d3904104\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw"
Jul 30 10:38:07.629026 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:07.628995    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw"
Jul 30 10:38:07.761041 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:07.761015    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw"]
Jul 30 10:38:07.763183 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:38:07.763153    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f42bfb2_0159_44cb_8538_6061d3904104.slice/crio-964448d8b189419cd498855b809d39bdd16323eb22d1a20757867f78d512e346 WatchSource:0}: Error finding container 964448d8b189419cd498855b809d39bdd16323eb22d1a20757867f78d512e346: Status 404 returned error can't find the container with id 964448d8b189419cd498855b809d39bdd16323eb22d1a20757867f78d512e346
Jul 30 10:38:08.033124 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:08.033089    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw" event={"ID":"1f42bfb2-0159-44cb-8538-6061d3904104","Type":"ContainerStarted","Data":"b2d72ab7e2a3fdcd921c03f7fa3dba83fe6f68fcb10873bbc15b60e1285d78b5"}
Jul 30 10:38:08.033294 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:08.033139    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw" event={"ID":"1f42bfb2-0159-44cb-8538-6061d3904104","Type":"ContainerStarted","Data":"964448d8b189419cd498855b809d39bdd16323eb22d1a20757867f78d512e346"}
Jul 30 10:38:08.050240 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:08.050193    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw" podStartSLOduration=1.050177506 podStartE2EDuration="1.050177506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:38:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:38:08.048020133 +0000 UTC m=+2924.542215639" watchObservedRunningTime="2026-07-30 10:38:08.050177506 +0000 UTC m=+2924.544372990"
Jul 30 10:38:09.038659 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:09.038620    2567 generic.go:356] "Generic (PLEG): container finished" podID="1f42bfb2-0159-44cb-8538-6061d3904104" containerID="b2d72ab7e2a3fdcd921c03f7fa3dba83fe6f68fcb10873bbc15b60e1285d78b5" exitCode=0
Jul 30 10:38:09.039062 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:09.038700    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw" event={"ID":"1f42bfb2-0159-44cb-8538-6061d3904104","Type":"ContainerDied","Data":"b2d72ab7e2a3fdcd921c03f7fa3dba83fe6f68fcb10873bbc15b60e1285d78b5"}
Jul 30 10:38:10.175752 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:10.175727    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw"
Jul 30 10:38:10.241090 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:10.241058    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f42bfb2-0159-44cb-8538-6061d3904104-maas-api-service-ca\") pod \"1f42bfb2-0159-44cb-8538-6061d3904104\" (UID: \"1f42bfb2-0159-44cb-8538-6061d3904104\") "
Jul 30 10:38:10.241373 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:10.241329    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f42bfb2-0159-44cb-8538-6061d3904104-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1f42bfb2-0159-44cb-8538-6061d3904104" (UID: "1f42bfb2-0159-44cb-8538-6061d3904104"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 10:38:10.342385 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:10.342277    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f42bfb2-0159-44cb-8538-6061d3904104-maas-api-service-ca\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:38:11.052361 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:11.048977    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw" event={"ID":"1f42bfb2-0159-44cb-8538-6061d3904104","Type":"ContainerDied","Data":"964448d8b189419cd498855b809d39bdd16323eb22d1a20757867f78d512e346"}
Jul 30 10:38:11.052361 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:11.049008    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw"
Jul 30 10:38:11.052361 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:11.049089    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="964448d8b189419cd498855b809d39bdd16323eb22d1a20757867f78d512e346"
Jul 30 10:38:23.021258 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:23.021225    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw"]
Jul 30 10:38:23.023719 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:23.023685    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-220161-6vjpw"]
Jul 30 10:38:24.063480 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:24.063448    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f42bfb2-0159-44cb-8538-6061d3904104" path="/var/lib/kubelet/pods/1f42bfb2-0159-44cb-8538-6061d3904104/volumes"
Jul 30 10:38:27.496567 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:27.496527    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:38:38.807815 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:38:38.807778    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:39:05.899860 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:05.899823    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:39:24.108291 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:24.108260    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:39:24.114252 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:24.114232    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:39:24.117174 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:24.117154    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:39:24.123000 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:24.122984    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:39:37.523310 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.523275    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"]
Jul 30 10:39:37.523730 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.523630    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f42bfb2-0159-44cb-8538-6061d3904104" containerName="revoke-keys"
Jul 30 10:39:37.523730 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.523643    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f42bfb2-0159-44cb-8538-6061d3904104" containerName="revoke-keys"
Jul 30 10:39:37.523730 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.523729    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f42bfb2-0159-44cb-8538-6061d3904104" containerName="revoke-keys"
Jul 30 10:39:37.526579 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.526560    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.529250 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.529226    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b75946\"/\"kube-root-ca.crt\""
Jul 30 10:39:37.529399 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.529274    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b75946\"/\"openshift-service-ca.crt\""
Jul 30 10:39:37.530385 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.530360    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b75946\"/\"default-dockercfg-m5nlh\""
Jul 30 10:39:37.530385 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.530382    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b75946\"/\"auth-test-model-b75946-kserve-self-signed-certs\""
Jul 30 10:39:37.535015 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.534991    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"]
Jul 30 10:39:37.581422 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.581390    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2791798-524b-4082-b8d8-abe7978da473-tls-certs\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.581571 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.581468    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-dshm\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.581571 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.581525    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtlgf\" (UniqueName: \"kubernetes.io/projected/b2791798-524b-4082-b8d8-abe7978da473-kube-api-access-jtlgf\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.581675 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.581621    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-model-cache\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.581675 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.581652    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-tmp-dir\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.581756 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.581675    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-home\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.682947 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.682916    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2791798-524b-4082-b8d8-abe7978da473-tls-certs\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.683128 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.682958    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-dshm\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.683128 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.682989    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtlgf\" (UniqueName: \"kubernetes.io/projected/b2791798-524b-4082-b8d8-abe7978da473-kube-api-access-jtlgf\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.683128 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.683043    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-model-cache\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.683128 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.683059    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-tmp-dir\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.683128 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.683076    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-home\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.683523 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.683498    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-model-cache\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.683523 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.683514    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-home\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.683642 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.683562    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-tmp-dir\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.685214 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.685187    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-dshm\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.685494 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.685473    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2791798-524b-4082-b8d8-abe7978da473-tls-certs\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.690678 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.690659    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtlgf\" (UniqueName: \"kubernetes.io/projected/b2791798-524b-4082-b8d8-abe7978da473-kube-api-access-jtlgf\") pod \"auth-test-model-b75946-kserve-846dfc6f99-7sjzx\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") " pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.836489 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.836393    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:37.992207 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:37.992171    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"]
Jul 30 10:39:37.995092 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:39:37.995066    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2791798_524b_4082_b8d8_abe7978da473.slice/crio-71b04b8155610c02c14c77526f407ad71ed2c1c79d9df26365f2ede902197a87 WatchSource:0}: Error finding container 71b04b8155610c02c14c77526f407ad71ed2c1c79d9df26365f2ede902197a87: Status 404 returned error can't find the container with id 71b04b8155610c02c14c77526f407ad71ed2c1c79d9df26365f2ede902197a87
Jul 30 10:39:38.437302 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:38.437266    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx" event={"ID":"b2791798-524b-4082-b8d8-abe7978da473","Type":"ContainerStarted","Data":"5c0cd6ebf962ae356344fab3ffc752a80f781ef748fcf0ca020624961d25355e"}
Jul 30 10:39:38.437302 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:38.437312    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx" event={"ID":"b2791798-524b-4082-b8d8-abe7978da473","Type":"ContainerStarted","Data":"71b04b8155610c02c14c77526f407ad71ed2c1c79d9df26365f2ede902197a87"}
Jul 30 10:39:38.452513 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:38.452368    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx" podStartSLOduration=1.452322851 podStartE2EDuration="1.452322851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:39:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:39:38.451630747 +0000 UTC m=+3014.945826268" watchObservedRunningTime="2026-07-30 10:39:38.452322851 +0000 UTC m=+3014.946518370"
Jul 30 10:39:41.088049 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:41.088007    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:39:47.836767 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:47.836729    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:47.836767 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:47.836777    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:47.849351 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:47.849299    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:48.495305 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:48.495270    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:39:59.511754 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.511721    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"]
Jul 30 10:39:59.515617 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.515599    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.518045 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.518023    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bdf164\"/\"kube-root-ca.crt\""
Jul 30 10:39:59.518172 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.518060    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bdf164\"/\"openshift-service-ca.crt\""
Jul 30 10:39:59.519244 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.519228    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bdf164\"/\"default-dockercfg-24rlr\""
Jul 30 10:39:59.519378 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.519290    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bdf164\"/\"auth-test-model-bdf164-kserve-self-signed-certs\""
Jul 30 10:39:59.524277 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.524258    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"]
Jul 30 10:39:59.687579 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.687542    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tmp-dir\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.687579 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.687578    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-dshm\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.687817 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.687639    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-home\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.687817 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.687725    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.687817 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.687765    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-model-cache\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.687817 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.687785    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8886n\" (UniqueName: \"kubernetes.io/projected/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-kube-api-access-8886n\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.789211 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.789126    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.789211 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.789179    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-model-cache\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.789444 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.789291    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8886n\" (UniqueName: \"kubernetes.io/projected/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-kube-api-access-8886n\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.789444 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.789385    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tmp-dir\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.789444 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.789418    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-dshm\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.789604 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.789470    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-home\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.789657 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.789629    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-model-cache\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.789761 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.789742    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tmp-dir\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.789846 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.789830    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-home\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.791607 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.791586    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-dshm\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.791799 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.791781    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.799057 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.799032    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8886n\" (UniqueName: \"kubernetes.io/projected/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-kube-api-access-8886n\") pod \"auth-test-model-bdf164-kserve-75d8cb489f-dxbfc\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") " pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.825851 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.825825    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:39:59.960413 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:39:59.960382    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"]
Jul 30 10:39:59.962186 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:39:59.962153    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9eedc55e_4ba2_4c1d_bd3a_a165fdbd3e27.slice/crio-ed6a9fada17a417717e0db1c953b52ec3bbb0b6f651252b0d69166c9a961056a WatchSource:0}: Error finding container ed6a9fada17a417717e0db1c953b52ec3bbb0b6f651252b0d69166c9a961056a: Status 404 returned error can't find the container with id ed6a9fada17a417717e0db1c953b52ec3bbb0b6f651252b0d69166c9a961056a
Jul 30 10:40:00.541133 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:40:00.541095    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc" event={"ID":"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27","Type":"ContainerStarted","Data":"fa521696db71843626754685ca46f10c7fd8c02cc00f670bcb2c7ae8fab3f3a9"}
Jul 30 10:40:00.541541 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:40:00.541146    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc" event={"ID":"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27","Type":"ContainerStarted","Data":"ed6a9fada17a417717e0db1c953b52ec3bbb0b6f651252b0d69166c9a961056a"}
Jul 30 10:40:00.560826 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:40:00.560776    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc" podStartSLOduration=1.560759146 podStartE2EDuration="1.560759146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:39:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:40:00.559297595 +0000 UTC m=+3037.053493114" watchObservedRunningTime="2026-07-30 10:40:00.560759146 +0000 UTC m=+3037.054954689"
Jul 30 10:40:03.699378 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:40:03.699327    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:40:09.826450 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:40:09.826409    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:40:09.826891 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:40:09.826466    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:40:09.838873 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:40:09.838846    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:40:10.599887 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:40:10.599856    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:40:27.401667 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:40:27.401595    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:40:53.500863 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:40:53.500825    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:41:10.999446 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:41:10.999412    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:41:32.108212 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:41:32.108174    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:41:58.199232 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:41:58.199138    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:42:16.804136 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:42:16.804098    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:42:41.990448 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:42:41.990412    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:42:59.697791 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:42:59.697758    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:43:27.391617 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:43:27.391528    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:43:44.491894 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:43:44.491862    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:44:12.295192 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:44:12.295157    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:44:24.135111 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:44:24.135081    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:44:24.141290 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:44:24.141269    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:44:24.145599 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:44:24.145582    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:44:24.151393 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:44:24.151376    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:44:24.221822 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:44:24.221790    2567 scope.go:117] "RemoveContainer" containerID="b2d72ab7e2a3fdcd921c03f7fa3dba83fe6f68fcb10873bbc15b60e1285d78b5"
Jul 30 10:44:24.229836 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:44:24.229815    2567 scope.go:117] "RemoveContainer" containerID="d21e604f3dbed4667b024867a610633d5b693b1d00fec52e285fe330f614a627"
Jul 30 10:44:28.808135 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:44:28.808097    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:44:50.002711 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:44:50.002631    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:45:00.140614 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.140569    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756805-zwx9k"]
Jul 30 10:45:00.144328 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.144302    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bdf164-29756805-57jzd"]
Jul 30 10:45:00.144485 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.144422    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756805-zwx9k"
Jul 30 10:45:00.146858 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.146836    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j6dj6\""
Jul 30 10:45:00.147494 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.147474    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bdf164-29756805-57jzd"
Jul 30 10:45:00.154445 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.154428    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756805-zwx9k"
Jul 30 10:45:00.159804 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.159772    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756805-zwx9k"]
Jul 30 10:45:00.162578 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.162556    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bdf164-29756805-57jzd"
Jul 30 10:45:00.162840 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.162820    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bdf164-29756805-57jzd"]
Jul 30 10:45:00.300501 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.300471    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756805-zwx9k"]
Jul 30 10:45:00.302671 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:45:00.302636    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b46a103_6400_40c8_987d_a82fa502692f.slice/crio-448ead6ed7b394326c4926db3b99f0e84cabfbbaed99fd90e3e698539b22f94c WatchSource:0}: Error finding container 448ead6ed7b394326c4926db3b99f0e84cabfbbaed99fd90e3e698539b22f94c: Status 404 returned error can't find the container with id 448ead6ed7b394326c4926db3b99f0e84cabfbbaed99fd90e3e698539b22f94c
Jul 30 10:45:00.319915 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.319895    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bdf164-29756805-57jzd"]
Jul 30 10:45:00.322520 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:45:00.322498    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c48ac37_c1d2_4c84_8399_ed075168cdda.slice/crio-a048288a2b3513dc6ffda277823e6e563490f01bb2b6b2108d5ab3dacfd8bbcc WatchSource:0}: Error finding container a048288a2b3513dc6ffda277823e6e563490f01bb2b6b2108d5ab3dacfd8bbcc: Status 404 returned error can't find the container with id a048288a2b3513dc6ffda277823e6e563490f01bb2b6b2108d5ab3dacfd8bbcc
Jul 30 10:45:00.911719 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.911687    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bdf164-29756805-57jzd" event={"ID":"4c48ac37-c1d2-4c84-8399-ed075168cdda","Type":"ContainerStarted","Data":"d00fd6db8bfab1ba690785ffe8518f3744562e0242f7989e9d3b2a7feb21df50"}
Jul 30 10:45:00.911917 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.911742    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bdf164-29756805-57jzd" event={"ID":"4c48ac37-c1d2-4c84-8399-ed075168cdda","Type":"ContainerStarted","Data":"a048288a2b3513dc6ffda277823e6e563490f01bb2b6b2108d5ab3dacfd8bbcc"}
Jul 30 10:45:00.913212 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.913191    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756805-zwx9k" event={"ID":"6b46a103-6400-40c8-987d-a82fa502692f","Type":"ContainerStarted","Data":"cfc496c659fcb12b92f0f5d3840a2b8cdb4ffeeee1ddca21af9faf0a38eb85a7"}
Jul 30 10:45:00.913323 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.913257    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756805-zwx9k" event={"ID":"6b46a103-6400-40c8-987d-a82fa502692f","Type":"ContainerStarted","Data":"448ead6ed7b394326c4926db3b99f0e84cabfbbaed99fd90e3e698539b22f94c"}
Jul 30 10:45:00.927815 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.927776    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bdf164-29756805-57jzd" podStartSLOduration=0.927761468 podStartE2EDuration="927.761468ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:45:00.925773871 +0000 UTC m=+3337.419969379" watchObservedRunningTime="2026-07-30 10:45:00.927761468 +0000 UTC m=+3337.421956973"
Jul 30 10:45:00.940209 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:00.940168    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756805-zwx9k" podStartSLOduration=0.940156039 podStartE2EDuration="940.156039ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:45:00.939723435 +0000 UTC m=+3337.433918942" watchObservedRunningTime="2026-07-30 10:45:00.940156039 +0000 UTC m=+3337.434351544"
Jul 30 10:45:01.919388 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:01.919329    2567 generic.go:356] "Generic (PLEG): container finished" podID="4c48ac37-c1d2-4c84-8399-ed075168cdda" containerID="d00fd6db8bfab1ba690785ffe8518f3744562e0242f7989e9d3b2a7feb21df50" exitCode=0
Jul 30 10:45:01.919850 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:01.919440    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bdf164-29756805-57jzd" event={"ID":"4c48ac37-c1d2-4c84-8399-ed075168cdda","Type":"ContainerDied","Data":"d00fd6db8bfab1ba690785ffe8518f3744562e0242f7989e9d3b2a7feb21df50"}
Jul 30 10:45:01.920909 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:01.920888    2567 generic.go:356] "Generic (PLEG): container finished" podID="6b46a103-6400-40c8-987d-a82fa502692f" containerID="cfc496c659fcb12b92f0f5d3840a2b8cdb4ffeeee1ddca21af9faf0a38eb85a7" exitCode=0
Jul 30 10:45:01.921002 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:01.920923    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756805-zwx9k" event={"ID":"6b46a103-6400-40c8-987d-a82fa502692f","Type":"ContainerDied","Data":"cfc496c659fcb12b92f0f5d3840a2b8cdb4ffeeee1ddca21af9faf0a38eb85a7"}
Jul 30 10:45:03.082785 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:03.082757    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bdf164-29756805-57jzd"
Jul 30 10:45:03.085966 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:03.085945    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756805-zwx9k"
Jul 30 10:45:03.932484 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:03.932455    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bdf164-29756805-57jzd"
Jul 30 10:45:03.932672 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:03.932487    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bdf164-29756805-57jzd" event={"ID":"4c48ac37-c1d2-4c84-8399-ed075168cdda","Type":"ContainerDied","Data":"a048288a2b3513dc6ffda277823e6e563490f01bb2b6b2108d5ab3dacfd8bbcc"}
Jul 30 10:45:03.932672 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:03.932537    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a048288a2b3513dc6ffda277823e6e563490f01bb2b6b2108d5ab3dacfd8bbcc"
Jul 30 10:45:03.933637 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:03.933621    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756805-zwx9k"
Jul 30 10:45:03.933741 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:03.933649    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756805-zwx9k" event={"ID":"6b46a103-6400-40c8-987d-a82fa502692f","Type":"ContainerDied","Data":"448ead6ed7b394326c4926db3b99f0e84cabfbbaed99fd90e3e698539b22f94c"}
Jul 30 10:45:03.933741 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:03.933690    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="448ead6ed7b394326c4926db3b99f0e84cabfbbaed99fd90e3e698539b22f94c"
Jul 30 10:45:15.600378 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:15.600322    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:45:19.339082 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:19.339046    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"]
Jul 30 10:45:20.211139 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:20.211106    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"]
Jul 30 10:45:20.221607 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:45:20.221576    2567 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-self-signed-certs: secret "auth-test-model-bdf164-kserve-self-signed-certs" not found
Jul 30 10:45:20.221743 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:45:20.221656    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs podName:9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27 nodeName:}" failed. No retries permitted until 2026-07-30 10:45:20.721633977 +0000 UTC m=+3357.215829470 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs") pod "auth-test-model-bdf164-kserve-75d8cb489f-dxbfc" (UID: "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27") : secret "auth-test-model-bdf164-kserve-self-signed-certs" not found
Jul 30 10:45:20.725780 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:45:20.725745    2567 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-self-signed-certs: secret "auth-test-model-bdf164-kserve-self-signed-certs" not found
Jul 30 10:45:20.726189 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:45:20.725811    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs podName:9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27 nodeName:}" failed. No retries permitted until 2026-07-30 10:45:21.725795879 +0000 UTC m=+3358.219991363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs") pod "auth-test-model-bdf164-kserve-75d8cb489f-dxbfc" (UID: "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27") : secret "auth-test-model-bdf164-kserve-self-signed-certs" not found
Jul 30 10:45:21.733280 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:45:21.733244    2567 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-self-signed-certs: secret "auth-test-model-bdf164-kserve-self-signed-certs" not found
Jul 30 10:45:21.733706 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:45:21.733316    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs podName:9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27 nodeName:}" failed. No retries permitted until 2026-07-30 10:45:23.733300719 +0000 UTC m=+3360.227496203 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs") pod "auth-test-model-bdf164-kserve-75d8cb489f-dxbfc" (UID: "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27") : secret "auth-test-model-bdf164-kserve-self-signed-certs" not found
Jul 30 10:45:23.755122 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:45:23.755070    2567 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-self-signed-certs: secret "auth-test-model-bdf164-kserve-self-signed-certs" not found
Jul 30 10:45:23.755653 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:45:23.755150    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs podName:9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27 nodeName:}" failed. No retries permitted until 2026-07-30 10:45:27.755135956 +0000 UTC m=+3364.249331440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs") pod "auth-test-model-bdf164-kserve-75d8cb489f-dxbfc" (UID: "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27") : secret "auth-test-model-bdf164-kserve-self-signed-certs" not found
Jul 30 10:45:27.488281 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:27.488243    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:45:27.792395 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:45:27.792281    2567 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-self-signed-certs: secret "auth-test-model-bdf164-kserve-self-signed-certs" not found
Jul 30 10:45:27.792395 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:45:27.792383    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs podName:9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27 nodeName:}" failed. No retries permitted until 2026-07-30 10:45:35.792366636 +0000 UTC m=+3372.286562119 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs") pod "auth-test-model-bdf164-kserve-75d8cb489f-dxbfc" (UID: "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27") : secret "auth-test-model-bdf164-kserve-self-signed-certs" not found
Jul 30 10:45:34.358506 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.358440    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx" podUID="b2791798-524b-4082-b8d8-abe7978da473" containerName="main" containerID="cri-o://5c0cd6ebf962ae356344fab3ffc752a80f781ef748fcf0ca020624961d25355e" gracePeriod=45
Jul 30 10:45:34.618804 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.618744    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:45:34.660671 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.660640    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtlgf\" (UniqueName: \"kubernetes.io/projected/b2791798-524b-4082-b8d8-abe7978da473-kube-api-access-jtlgf\") pod \"b2791798-524b-4082-b8d8-abe7978da473\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") "
Jul 30 10:45:34.660864 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.660688    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-model-cache\") pod \"b2791798-524b-4082-b8d8-abe7978da473\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") "
Jul 30 10:45:34.660864 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.660727    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-dshm\") pod \"b2791798-524b-4082-b8d8-abe7978da473\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") "
Jul 30 10:45:34.660864 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.660793    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-tmp-dir\") pod \"b2791798-524b-4082-b8d8-abe7978da473\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") "
Jul 30 10:45:34.660864 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.660837    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-home\") pod \"b2791798-524b-4082-b8d8-abe7978da473\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") "
Jul 30 10:45:34.661075 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.660864    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2791798-524b-4082-b8d8-abe7978da473-tls-certs\") pod \"b2791798-524b-4082-b8d8-abe7978da473\" (UID: \"b2791798-524b-4082-b8d8-abe7978da473\") "
Jul 30 10:45:34.661127 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.661063    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-model-cache" (OuterVolumeSpecName: "model-cache") pod "b2791798-524b-4082-b8d8-abe7978da473" (UID: "b2791798-524b-4082-b8d8-abe7978da473"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:45:34.661127 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.661107    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b2791798-524b-4082-b8d8-abe7978da473" (UID: "b2791798-524b-4082-b8d8-abe7978da473"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:45:34.661299 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.661280    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-home" (OuterVolumeSpecName: "home") pod "b2791798-524b-4082-b8d8-abe7978da473" (UID: "b2791798-524b-4082-b8d8-abe7978da473"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:45:34.663038 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.663006    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-dshm" (OuterVolumeSpecName: "dshm") pod "b2791798-524b-4082-b8d8-abe7978da473" (UID: "b2791798-524b-4082-b8d8-abe7978da473"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:45:34.663150 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.663069    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2791798-524b-4082-b8d8-abe7978da473-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b2791798-524b-4082-b8d8-abe7978da473" (UID: "b2791798-524b-4082-b8d8-abe7978da473"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 10:45:34.663150 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.663073    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2791798-524b-4082-b8d8-abe7978da473-kube-api-access-jtlgf" (OuterVolumeSpecName: "kube-api-access-jtlgf") pod "b2791798-524b-4082-b8d8-abe7978da473" (UID: "b2791798-524b-4082-b8d8-abe7978da473"). InnerVolumeSpecName "kube-api-access-jtlgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:45:34.761759 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.761711    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:45:34.761759 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.761755    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:45:34.761759 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.761765    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2791798-524b-4082-b8d8-abe7978da473-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:45:34.761759 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.761774    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtlgf\" (UniqueName: \"kubernetes.io/projected/b2791798-524b-4082-b8d8-abe7978da473-kube-api-access-jtlgf\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:45:34.762055 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.761785    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:45:34.762055 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:34.761794    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2791798-524b-4082-b8d8-abe7978da473-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:45:35.076813 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:35.076776    2567 generic.go:356] "Generic (PLEG): container finished" podID="b2791798-524b-4082-b8d8-abe7978da473" containerID="5c0cd6ebf962ae356344fab3ffc752a80f781ef748fcf0ca020624961d25355e" exitCode=0
Jul 30 10:45:35.077000 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:35.076830    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"
Jul 30 10:45:35.077000 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:35.076859    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx" event={"ID":"b2791798-524b-4082-b8d8-abe7978da473","Type":"ContainerDied","Data":"5c0cd6ebf962ae356344fab3ffc752a80f781ef748fcf0ca020624961d25355e"}
Jul 30 10:45:35.077000 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:35.076904    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx" event={"ID":"b2791798-524b-4082-b8d8-abe7978da473","Type":"ContainerDied","Data":"71b04b8155610c02c14c77526f407ad71ed2c1c79d9df26365f2ede902197a87"}
Jul 30 10:45:35.077000 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:35.076921    2567 scope.go:117] "RemoveContainer" containerID="5c0cd6ebf962ae356344fab3ffc752a80f781ef748fcf0ca020624961d25355e"
Jul 30 10:45:35.086531 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:35.086512    2567 scope.go:117] "RemoveContainer" containerID="5c0cd6ebf962ae356344fab3ffc752a80f781ef748fcf0ca020624961d25355e"
Jul 30 10:45:35.086808 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:45:35.086789    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c0cd6ebf962ae356344fab3ffc752a80f781ef748fcf0ca020624961d25355e\": container with ID starting with 5c0cd6ebf962ae356344fab3ffc752a80f781ef748fcf0ca020624961d25355e not found: ID does not exist" containerID="5c0cd6ebf962ae356344fab3ffc752a80f781ef748fcf0ca020624961d25355e"
Jul 30 10:45:35.086870 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:35.086821    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c0cd6ebf962ae356344fab3ffc752a80f781ef748fcf0ca020624961d25355e"} err="failed to get container status \"5c0cd6ebf962ae356344fab3ffc752a80f781ef748fcf0ca020624961d25355e\": rpc error: code = NotFound desc = could not find container \"5c0cd6ebf962ae356344fab3ffc752a80f781ef748fcf0ca020624961d25355e\": container with ID starting with 5c0cd6ebf962ae356344fab3ffc752a80f781ef748fcf0ca020624961d25355e not found: ID does not exist"
Jul 30 10:45:35.098128 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:35.098095    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"]
Jul 30 10:45:35.104757 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:35.104732    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/auth-test-model-b75946-kserve-846dfc6f99-7sjzx"]
Jul 30 10:45:35.873717 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:45:35.873683    2567 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-self-signed-certs: secret "auth-test-model-bdf164-kserve-self-signed-certs" not found
Jul 30 10:45:35.874129 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:45:35.873752    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs podName:9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27 nodeName:}" failed. No retries permitted until 2026-07-30 10:45:51.873737277 +0000 UTC m=+3388.367932761 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs") pod "auth-test-model-bdf164-kserve-75d8cb489f-dxbfc" (UID: "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27") : secret "auth-test-model-bdf164-kserve-self-signed-certs" not found
Jul 30 10:45:36.030358 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.030304    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc" podUID="9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27" containerName="main" containerID="cri-o://fa521696db71843626754685ca46f10c7fd8c02cc00f670bcb2c7ae8fab3f3a9" gracePeriod=45
Jul 30 10:45:36.062909 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.062880    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2791798-524b-4082-b8d8-abe7978da473" path="/var/lib/kubelet/pods/b2791798-524b-4082-b8d8-abe7978da473/volumes"
Jul 30 10:45:36.288131 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.288105    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:45:36.378962 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.378932    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs\") pod \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") "
Jul 30 10:45:36.379134 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.378972    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-dshm\") pod \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") "
Jul 30 10:45:36.379134 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.378997    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tmp-dir\") pod \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") "
Jul 30 10:45:36.379134 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.379014    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8886n\" (UniqueName: \"kubernetes.io/projected/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-kube-api-access-8886n\") pod \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") "
Jul 30 10:45:36.379134 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.379058    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-model-cache\") pod \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") "
Jul 30 10:45:36.379134 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.379109    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-home\") pod \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\" (UID: \"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27\") "
Jul 30 10:45:36.379412 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.379274    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27" (UID: "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:45:36.379673 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.379406    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:45:36.379673 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.379399    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-model-cache" (OuterVolumeSpecName: "model-cache") pod "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27" (UID: "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:45:36.379673 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.379435    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-home" (OuterVolumeSpecName: "home") pod "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27" (UID: "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:45:36.381315 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.381292    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-kube-api-access-8886n" (OuterVolumeSpecName: "kube-api-access-8886n") pod "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27" (UID: "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27"). InnerVolumeSpecName "kube-api-access-8886n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:45:36.381315 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.381299    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-dshm" (OuterVolumeSpecName: "dshm") pod "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27" (UID: "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:45:36.381478 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.381350    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27" (UID: "9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 10:45:36.480309 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.480213    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:45:36.480309 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.480244    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:45:36.480309 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.480259    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:45:36.480309 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.480269    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:45:36.480309 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:36.480280    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8886n\" (UniqueName: \"kubernetes.io/projected/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27-kube-api-access-8886n\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:45:37.087890 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:37.087851    2567 generic.go:356] "Generic (PLEG): container finished" podID="9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27" containerID="fa521696db71843626754685ca46f10c7fd8c02cc00f670bcb2c7ae8fab3f3a9" exitCode=0
Jul 30 10:45:37.087890 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:37.087895    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"
Jul 30 10:45:37.088400 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:37.087907    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc" event={"ID":"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27","Type":"ContainerDied","Data":"fa521696db71843626754685ca46f10c7fd8c02cc00f670bcb2c7ae8fab3f3a9"}
Jul 30 10:45:37.088400 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:37.087945    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc" event={"ID":"9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27","Type":"ContainerDied","Data":"ed6a9fada17a417717e0db1c953b52ec3bbb0b6f651252b0d69166c9a961056a"}
Jul 30 10:45:37.088400 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:37.087968    2567 scope.go:117] "RemoveContainer" containerID="fa521696db71843626754685ca46f10c7fd8c02cc00f670bcb2c7ae8fab3f3a9"
Jul 30 10:45:37.096707 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:37.096690    2567 scope.go:117] "RemoveContainer" containerID="fa521696db71843626754685ca46f10c7fd8c02cc00f670bcb2c7ae8fab3f3a9"
Jul 30 10:45:37.097002 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:45:37.096982    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa521696db71843626754685ca46f10c7fd8c02cc00f670bcb2c7ae8fab3f3a9\": container with ID starting with fa521696db71843626754685ca46f10c7fd8c02cc00f670bcb2c7ae8fab3f3a9 not found: ID does not exist" containerID="fa521696db71843626754685ca46f10c7fd8c02cc00f670bcb2c7ae8fab3f3a9"
Jul 30 10:45:37.097055 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:37.097011    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa521696db71843626754685ca46f10c7fd8c02cc00f670bcb2c7ae8fab3f3a9"} err="failed to get container status \"fa521696db71843626754685ca46f10c7fd8c02cc00f670bcb2c7ae8fab3f3a9\": rpc error: code = NotFound desc = could not find container \"fa521696db71843626754685ca46f10c7fd8c02cc00f670bcb2c7ae8fab3f3a9\": container with ID starting with fa521696db71843626754685ca46f10c7fd8c02cc00f670bcb2c7ae8fab3f3a9 not found: ID does not exist"
Jul 30 10:45:37.108626 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:37.108596    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"]
Jul 30 10:45:37.112591 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:37.112569    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bdf164/auth-test-model-bdf164-kserve-75d8cb489f-dxbfc"]
Jul 30 10:45:38.064495 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:38.064466    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27" path="/var/lib/kubelet/pods/9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27/volumes"
Jul 30 10:45:50.415709 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.415675    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"]
Jul 30 10:45:50.416072 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.416036    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b46a103-6400-40c8-987d-a82fa502692f" containerName="cleanup"
Jul 30 10:45:50.416072 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.416048    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b46a103-6400-40c8-987d-a82fa502692f" containerName="cleanup"
Jul 30 10:45:50.416072 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.416056    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27" containerName="main"
Jul 30 10:45:50.416072 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.416062    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27" containerName="main"
Jul 30 10:45:50.416216 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.416077    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c48ac37-c1d2-4c84-8399-ed075168cdda" containerName="cleanup"
Jul 30 10:45:50.416216 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.416083    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c48ac37-c1d2-4c84-8399-ed075168cdda" containerName="cleanup"
Jul 30 10:45:50.416216 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.416089    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2791798-524b-4082-b8d8-abe7978da473" containerName="main"
Jul 30 10:45:50.416216 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.416098    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2791798-524b-4082-b8d8-abe7978da473" containerName="main"
Jul 30 10:45:50.416216 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.416156    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9eedc55e-4ba2-4c1d-bd3a-a165fdbd3e27" containerName="main"
Jul 30 10:45:50.416216 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.416163    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c48ac37-c1d2-4c84-8399-ed075168cdda" containerName="cleanup"
Jul 30 10:45:50.416216 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.416175    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2791798-524b-4082-b8d8-abe7978da473" containerName="main"
Jul 30 10:45:50.416216 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.416186    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b46a103-6400-40c8-987d-a82fa502692f" containerName="cleanup"
Jul 30 10:45:50.419169 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.419153    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.423058 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.423031    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b75946\"/\"openshift-service-ca.crt\""
Jul 30 10:45:50.423058 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.423048    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b75946\"/\"kube-root-ca.crt\""
Jul 30 10:45:50.423258 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.423040    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b75946\"/\"default-dockercfg-m5nlh\""
Jul 30 10:45:50.423258 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.423043    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b75946\"/\"sub-test-model-b75946-kserve-self-signed-certs\""
Jul 30 10:45:50.427880 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.427854    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"]
Jul 30 10:45:50.505508 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.505472    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-tmp-dir\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.505508 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.505509    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-dshm\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.505719 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.505527    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b7xw\" (UniqueName: \"kubernetes.io/projected/dab3cce3-b114-4264-9a2f-f726ffd4dca1-kube-api-access-6b7xw\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.505719 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.505642    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dab3cce3-b114-4264-9a2f-f726ffd4dca1-tls-certs\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.505719 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.505676    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-home\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.505719 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.505699    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-model-cache\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.606584 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.606542    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-tmp-dir\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.606584 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.606585    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-dshm\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.606852 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.606612    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b7xw\" (UniqueName: \"kubernetes.io/projected/dab3cce3-b114-4264-9a2f-f726ffd4dca1-kube-api-access-6b7xw\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.606852 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.606678    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dab3cce3-b114-4264-9a2f-f726ffd4dca1-tls-certs\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.606852 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.606706    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-home\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.606852 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.606734    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-model-cache\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.607128 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.607101    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-tmp-dir\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.607189 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.607142    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-home\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.607240 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.607183    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-model-cache\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.608971 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.608945    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-dshm\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.609213 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.609196    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dab3cce3-b114-4264-9a2f-f726ffd4dca1-tls-certs\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.614489 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.614468    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b7xw\" (UniqueName: \"kubernetes.io/projected/dab3cce3-b114-4264-9a2f-f726ffd4dca1-kube-api-access-6b7xw\") pod \"sub-test-model-b75946-kserve-788c477fcc-s99bz\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") " pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.729347 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.729242    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:45:50.866011 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:50.865980    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"]
Jul 30 10:45:50.867893 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:45:50.867867    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddab3cce3_b114_4264_9a2f_f726ffd4dca1.slice/crio-0de848477c0fa7d1beb6ff69f058632df5bcc0bd93b708772a6131551e5f93c6 WatchSource:0}: Error finding container 0de848477c0fa7d1beb6ff69f058632df5bcc0bd93b708772a6131551e5f93c6: Status 404 returned error can't find the container with id 0de848477c0fa7d1beb6ff69f058632df5bcc0bd93b708772a6131551e5f93c6
Jul 30 10:45:51.155446 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:51.155399    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz" event={"ID":"dab3cce3-b114-4264-9a2f-f726ffd4dca1","Type":"ContainerStarted","Data":"7cac2c01309dd0540605f4fbd1a8d4c7108c66c2ce064669b6c7c3a66da9aeff"}
Jul 30 10:45:51.155612 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:51.155458    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz" event={"ID":"dab3cce3-b114-4264-9a2f-f726ffd4dca1","Type":"ContainerStarted","Data":"0de848477c0fa7d1beb6ff69f058632df5bcc0bd93b708772a6131551e5f93c6"}
Jul 30 10:45:51.173927 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:51.173878    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz" podStartSLOduration=1.17386338 podStartE2EDuration="1.17386338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:45:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:45:51.172367388 +0000 UTC m=+3387.666562895" watchObservedRunningTime="2026-07-30 10:45:51.17386338 +0000 UTC m=+3387.668058886"
Jul 30 10:45:53.892669 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:45:53.892637    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:46:00.729866 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:46:00.729817    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:46:00.730281 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:46:00.729882    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:46:00.742459 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:46:00.742420    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:46:01.213688 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:46:01.213653    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:46:22.793446 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:46:22.793365    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:47:15.198144 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:47:15.198110    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:47:32.500022 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:47:32.499986    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:48:06.598008 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:06.597920    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:48:12.139498 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:12.139465    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"]
Jul 30 10:48:20.619898 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.619860    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"]
Jul 30 10:48:20.622681 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.622665    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.625105 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.625085    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b75946\"/\"rate-test-model-b75946-kserve-self-signed-certs\""
Jul 30 10:48:20.632495 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.632473    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"]
Jul 30 10:48:20.706036 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.706000    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-tls-certs\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.706216 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.706082    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-model-cache\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.706216 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.706121    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-home\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.706216 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.706188    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-tmp-dir\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.706216 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.706208    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgzpl\" (UniqueName: \"kubernetes.io/projected/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-kube-api-access-mgzpl\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.706425 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.706237    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-dshm\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.807585 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.807547    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-model-cache\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.807585 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.807587    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-home\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.807879 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.807612    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-tmp-dir\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.807879 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.807631    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgzpl\" (UniqueName: \"kubernetes.io/projected/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-kube-api-access-mgzpl\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.807879 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.807653    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-dshm\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.807879 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.807675    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-tls-certs\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.808104 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.808046    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-home\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.808104 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.808059    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-tmp-dir\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.808104 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.808086    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-model-cache\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.810035 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.810013    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-dshm\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.810358 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.810319    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-tls-certs\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.816373 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.816324    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgzpl\" (UniqueName: \"kubernetes.io/projected/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-kube-api-access-mgzpl\") pod \"rate-test-model-b75946-kserve-5fdff645f5-8g47v\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") " pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:20.931421 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:20.931380    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:21.063838 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:21.063813    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"]
Jul 30 10:48:21.065815 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:48:21.065787    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a8826ae_1c2d_41e1_a0f1_8ae5627e6fce.slice/crio-f3092b7b57863908cc265a76ae52ad786a90c224ffa736344f460bceeed9d351 WatchSource:0}: Error finding container f3092b7b57863908cc265a76ae52ad786a90c224ffa736344f460bceeed9d351: Status 404 returned error can't find the container with id f3092b7b57863908cc265a76ae52ad786a90c224ffa736344f460bceeed9d351
Jul 30 10:48:21.839783 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:21.839749    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v" event={"ID":"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce","Type":"ContainerStarted","Data":"43a18241480bf43e7de12d62727f363a0b56cd1b1064a639cd3528f56ef8ca4b"}
Jul 30 10:48:21.840189 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:21.839792    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v" event={"ID":"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce","Type":"ContainerStarted","Data":"f3092b7b57863908cc265a76ae52ad786a90c224ffa736344f460bceeed9d351"}
Jul 30 10:48:21.857880 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:21.857818    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v" podStartSLOduration=1.8578002900000001 podStartE2EDuration="1.85780029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:48:21.856850314 +0000 UTC m=+3538.351045819" watchObservedRunningTime="2026-07-30 10:48:21.85780029 +0000 UTC m=+3538.351995795"
Jul 30 10:48:23.801207 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:23.801163    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:48:27.160190 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.160149    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz" podUID="dab3cce3-b114-4264-9a2f-f726ffd4dca1" containerName="main" containerID="cri-o://7cac2c01309dd0540605f4fbd1a8d4c7108c66c2ce064669b6c7c3a66da9aeff" gracePeriod=45
Jul 30 10:48:27.409773 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.409746    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:48:27.465947 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.465863    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b7xw\" (UniqueName: \"kubernetes.io/projected/dab3cce3-b114-4264-9a2f-f726ffd4dca1-kube-api-access-6b7xw\") pod \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") "
Jul 30 10:48:27.465947 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.465910    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dab3cce3-b114-4264-9a2f-f726ffd4dca1-tls-certs\") pod \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") "
Jul 30 10:48:27.465947 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.465939    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-model-cache\") pod \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") "
Jul 30 10:48:27.466221 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.465966    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-dshm\") pod \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") "
Jul 30 10:48:27.466221 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.465991    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-tmp-dir\") pod \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") "
Jul 30 10:48:27.466221 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.466016    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-home\") pod \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\" (UID: \"dab3cce3-b114-4264-9a2f-f726ffd4dca1\") "
Jul 30 10:48:27.466400 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.466192    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-model-cache" (OuterVolumeSpecName: "model-cache") pod "dab3cce3-b114-4264-9a2f-f726ffd4dca1" (UID: "dab3cce3-b114-4264-9a2f-f726ffd4dca1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:48:27.466400 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.466230    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dab3cce3-b114-4264-9a2f-f726ffd4dca1" (UID: "dab3cce3-b114-4264-9a2f-f726ffd4dca1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:48:27.466400 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.466368    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:48:27.466400 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.466387    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:48:27.466575 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.466410    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-home" (OuterVolumeSpecName: "home") pod "dab3cce3-b114-4264-9a2f-f726ffd4dca1" (UID: "dab3cce3-b114-4264-9a2f-f726ffd4dca1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:48:27.468142 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.468123    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-dshm" (OuterVolumeSpecName: "dshm") pod "dab3cce3-b114-4264-9a2f-f726ffd4dca1" (UID: "dab3cce3-b114-4264-9a2f-f726ffd4dca1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:48:27.468142 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.468128    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dab3cce3-b114-4264-9a2f-f726ffd4dca1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dab3cce3-b114-4264-9a2f-f726ffd4dca1" (UID: "dab3cce3-b114-4264-9a2f-f726ffd4dca1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 10:48:27.468296 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.468171    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dab3cce3-b114-4264-9a2f-f726ffd4dca1-kube-api-access-6b7xw" (OuterVolumeSpecName: "kube-api-access-6b7xw") pod "dab3cce3-b114-4264-9a2f-f726ffd4dca1" (UID: "dab3cce3-b114-4264-9a2f-f726ffd4dca1"). InnerVolumeSpecName "kube-api-access-6b7xw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:48:27.567758 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.567717    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b7xw\" (UniqueName: \"kubernetes.io/projected/dab3cce3-b114-4264-9a2f-f726ffd4dca1-kube-api-access-6b7xw\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:48:27.567758 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.567751    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dab3cce3-b114-4264-9a2f-f726ffd4dca1-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:48:27.567758 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.567760    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:48:27.567758 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.567768    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dab3cce3-b114-4264-9a2f-f726ffd4dca1-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:48:27.869088 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.868998    2567 generic.go:356] "Generic (PLEG): container finished" podID="dab3cce3-b114-4264-9a2f-f726ffd4dca1" containerID="7cac2c01309dd0540605f4fbd1a8d4c7108c66c2ce064669b6c7c3a66da9aeff" exitCode=0
Jul 30 10:48:27.869088 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.869043    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"
Jul 30 10:48:27.869265 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.869095    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz" event={"ID":"dab3cce3-b114-4264-9a2f-f726ffd4dca1","Type":"ContainerDied","Data":"7cac2c01309dd0540605f4fbd1a8d4c7108c66c2ce064669b6c7c3a66da9aeff"}
Jul 30 10:48:27.869265 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.869134    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz" event={"ID":"dab3cce3-b114-4264-9a2f-f726ffd4dca1","Type":"ContainerDied","Data":"0de848477c0fa7d1beb6ff69f058632df5bcc0bd93b708772a6131551e5f93c6"}
Jul 30 10:48:27.869265 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.869152    2567 scope.go:117] "RemoveContainer" containerID="7cac2c01309dd0540605f4fbd1a8d4c7108c66c2ce064669b6c7c3a66da9aeff"
Jul 30 10:48:27.878171 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.878153    2567 scope.go:117] "RemoveContainer" containerID="7cac2c01309dd0540605f4fbd1a8d4c7108c66c2ce064669b6c7c3a66da9aeff"
Jul 30 10:48:27.878448 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:48:27.878421    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cac2c01309dd0540605f4fbd1a8d4c7108c66c2ce064669b6c7c3a66da9aeff\": container with ID starting with 7cac2c01309dd0540605f4fbd1a8d4c7108c66c2ce064669b6c7c3a66da9aeff not found: ID does not exist" containerID="7cac2c01309dd0540605f4fbd1a8d4c7108c66c2ce064669b6c7c3a66da9aeff"
Jul 30 10:48:27.878535 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.878452    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cac2c01309dd0540605f4fbd1a8d4c7108c66c2ce064669b6c7c3a66da9aeff"} err="failed to get container status \"7cac2c01309dd0540605f4fbd1a8d4c7108c66c2ce064669b6c7c3a66da9aeff\": rpc error: code = NotFound desc = could not find container \"7cac2c01309dd0540605f4fbd1a8d4c7108c66c2ce064669b6c7c3a66da9aeff\": container with ID starting with 7cac2c01309dd0540605f4fbd1a8d4c7108c66c2ce064669b6c7c3a66da9aeff not found: ID does not exist"
Jul 30 10:48:27.889765 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.889742    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"]
Jul 30 10:48:27.896266 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:27.896245    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/sub-test-model-b75946-kserve-788c477fcc-s99bz"]
Jul 30 10:48:28.063432 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:28.063396    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dab3cce3-b114-4264-9a2f-f726ffd4dca1" path="/var/lib/kubelet/pods/dab3cce3-b114-4264-9a2f-f726ffd4dca1/volumes"
Jul 30 10:48:30.931632 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:30.931604    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:30.932006 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:30.931652    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:30.944034 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:30.944006    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:31.898896 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:31.898866    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:48:46.413972 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.413940    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"]
Jul 30 10:48:46.414596 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.414540    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dab3cce3-b114-4264-9a2f-f726ffd4dca1" containerName="main"
Jul 30 10:48:46.414596 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.414561    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dab3cce3-b114-4264-9a2f-f726ffd4dca1" containerName="main"
Jul 30 10:48:46.414723 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.414686    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dab3cce3-b114-4264-9a2f-f726ffd4dca1" containerName="main"
Jul 30 10:48:46.417604 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.417584    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.420438 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.420410    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bdf164\"/\"default-dockercfg-24rlr\""
Jul 30 10:48:46.420531 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.420449    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bdf164\"/\"kube-root-ca.crt\""
Jul 30 10:48:46.421226 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.421211    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bdf164\"/\"openshift-service-ca.crt\""
Jul 30 10:48:46.421425 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.421409    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bdf164\"/\"rate-test-model-bdf164-kserve-self-signed-certs\""
Jul 30 10:48:46.428523 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.428502    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"]
Jul 30 10:48:46.537709 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.537659    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-model-cache\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.537709 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.537711    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlb8p\" (UniqueName: \"kubernetes.io/projected/9b282240-735f-4372-867c-ffeac9971cee-kube-api-access-qlb8p\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.537932 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.537743    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-dshm\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.537932 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.537795    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-home\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.537932 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.537856    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b282240-735f-4372-867c-ffeac9971cee-tls-certs\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.537932 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.537923    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-tmp-dir\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.639012 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.638972    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-tmp-dir\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.639012 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.639026    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-model-cache\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.639285 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.639049    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlb8p\" (UniqueName: \"kubernetes.io/projected/9b282240-735f-4372-867c-ffeac9971cee-kube-api-access-qlb8p\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.639285 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.639070    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-dshm\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.639285 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.639089    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-home\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.639285 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.639134    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b282240-735f-4372-867c-ffeac9971cee-tls-certs\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.639557 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.639502    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-model-cache\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.639557 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.639515    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-home\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.639632 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.639562    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-tmp-dir\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.641463 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.641442    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-dshm\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.641598 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.641581    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b282240-735f-4372-867c-ffeac9971cee-tls-certs\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.646895 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.646867    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlb8p\" (UniqueName: \"kubernetes.io/projected/9b282240-735f-4372-867c-ffeac9971cee-kube-api-access-qlb8p\") pod \"rate-test-model-bdf164-kserve-8685d96855-x2tfr\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") " pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.727000 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.726900    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:46.861043 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.861017    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"]
Jul 30 10:48:46.862906 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:48:46.862875    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b282240_735f_4372_867c_ffeac9971cee.slice/crio-c6e95a77fcedc8fbc2ac89033780ff37b1b7d7fb99dac82682d6ca9d11254eb0 WatchSource:0}: Error finding container c6e95a77fcedc8fbc2ac89033780ff37b1b7d7fb99dac82682d6ca9d11254eb0: Status 404 returned error can't find the container with id c6e95a77fcedc8fbc2ac89033780ff37b1b7d7fb99dac82682d6ca9d11254eb0
Jul 30 10:48:46.965511 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.965475    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr" event={"ID":"9b282240-735f-4372-867c-ffeac9971cee","Type":"ContainerStarted","Data":"ad4798c55e928607dce242097f299f1dee4fe0a39a3ea5d36dde5b8f1b1f29cc"}
Jul 30 10:48:46.965691 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.965520    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr" event={"ID":"9b282240-735f-4372-867c-ffeac9971cee","Type":"ContainerStarted","Data":"c6e95a77fcedc8fbc2ac89033780ff37b1b7d7fb99dac82682d6ca9d11254eb0"}
Jul 30 10:48:46.984434 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:46.984311    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr" podStartSLOduration=0.984293443 podStartE2EDuration="984.293443ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:48:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:48:46.982209678 +0000 UTC m=+3563.476405182" watchObservedRunningTime="2026-07-30 10:48:46.984293443 +0000 UTC m=+3563.478488991"
Jul 30 10:48:49.790851 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:49.790805    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:48:56.727920 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:56.727872    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:56.728427 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:56.727946    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:56.740659 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:56.740629    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:48:57.022693 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:48:57.022605    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:49:24.161344 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:49:24.161302    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:49:24.167552 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:49:24.167525    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:49:24.175091 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:49:24.175071    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:49:24.180537 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:49:24.180518    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:49:27.309584 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:49:27.309553    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:49:44.495735 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:49:44.495693    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:50:19.594016 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:19.593977    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:50:25.361424 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:25.361386    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"]
Jul 30 10:50:26.191987 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:26.191955    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"]
Jul 30 10:50:36.495931 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:36.495896    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:50:40.380020 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.379951    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v" podUID="7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce" containerName="main" containerID="cri-o://43a18241480bf43e7de12d62727f363a0b56cd1b1064a639cd3528f56ef8ca4b" gracePeriod=45
Jul 30 10:50:40.636829 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.636760    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:50:40.721853 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.721819    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-dshm\") pod \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") "
Jul 30 10:50:40.722038 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.721882    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-tls-certs\") pod \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") "
Jul 30 10:50:40.722038 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.721945    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-tmp-dir\") pod \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") "
Jul 30 10:50:40.722038 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.721982    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-model-cache\") pod \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") "
Jul 30 10:50:40.722038 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.722000    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-home\") pod \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") "
Jul 30 10:50:40.722212 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.722079    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgzpl\" (UniqueName: \"kubernetes.io/projected/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-kube-api-access-mgzpl\") pod \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\" (UID: \"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce\") "
Jul 30 10:50:40.722290 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.722260    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce" (UID: "7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:50:40.722377 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.722283    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce" (UID: "7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:50:40.722377 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.722314    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-home" (OuterVolumeSpecName: "home") pod "7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce" (UID: "7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:50:40.724102 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.724076    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce" (UID: "7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 10:50:40.724102 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.724079    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-dshm" (OuterVolumeSpecName: "dshm") pod "7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce" (UID: "7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:50:40.724252 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.724185    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-kube-api-access-mgzpl" (OuterVolumeSpecName: "kube-api-access-mgzpl") pod "7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce" (UID: "7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce"). InnerVolumeSpecName "kube-api-access-mgzpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:50:40.822920 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.822890    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgzpl\" (UniqueName: \"kubernetes.io/projected/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-kube-api-access-mgzpl\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:50:40.822920 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.822917    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:50:40.822920 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.822928    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:50:40.823150 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.822936    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:50:40.823150 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.822944    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:50:40.823150 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:40.822955    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:50:41.210860 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.210793    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr" podUID="9b282240-735f-4372-867c-ffeac9971cee" containerName="main" containerID="cri-o://ad4798c55e928607dce242097f299f1dee4fe0a39a3ea5d36dde5b8f1b1f29cc" gracePeriod=45
Jul 30 10:50:41.460735 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.460704    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:50:41.484031 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.483946    2567 generic.go:356] "Generic (PLEG): container finished" podID="7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce" containerID="43a18241480bf43e7de12d62727f363a0b56cd1b1064a639cd3528f56ef8ca4b" exitCode=0
Jul 30 10:50:41.484031 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.484001    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"
Jul 30 10:50:41.484242 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.484020    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v" event={"ID":"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce","Type":"ContainerDied","Data":"43a18241480bf43e7de12d62727f363a0b56cd1b1064a639cd3528f56ef8ca4b"}
Jul 30 10:50:41.484242 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.484065    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v" event={"ID":"7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce","Type":"ContainerDied","Data":"f3092b7b57863908cc265a76ae52ad786a90c224ffa736344f460bceeed9d351"}
Jul 30 10:50:41.484242 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.484087    2567 scope.go:117] "RemoveContainer" containerID="43a18241480bf43e7de12d62727f363a0b56cd1b1064a639cd3528f56ef8ca4b"
Jul 30 10:50:41.485524 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.485499    2567 generic.go:356] "Generic (PLEG): container finished" podID="9b282240-735f-4372-867c-ffeac9971cee" containerID="ad4798c55e928607dce242097f299f1dee4fe0a39a3ea5d36dde5b8f1b1f29cc" exitCode=0
Jul 30 10:50:41.485656 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.485528    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr" event={"ID":"9b282240-735f-4372-867c-ffeac9971cee","Type":"ContainerDied","Data":"ad4798c55e928607dce242097f299f1dee4fe0a39a3ea5d36dde5b8f1b1f29cc"}
Jul 30 10:50:41.485656 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.485545    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"
Jul 30 10:50:41.485656 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.485577    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr" event={"ID":"9b282240-735f-4372-867c-ffeac9971cee","Type":"ContainerDied","Data":"c6e95a77fcedc8fbc2ac89033780ff37b1b7d7fb99dac82682d6ca9d11254eb0"}
Jul 30 10:50:41.494394 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.494376    2567 scope.go:117] "RemoveContainer" containerID="43a18241480bf43e7de12d62727f363a0b56cd1b1064a639cd3528f56ef8ca4b"
Jul 30 10:50:41.494657 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:50:41.494637    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43a18241480bf43e7de12d62727f363a0b56cd1b1064a639cd3528f56ef8ca4b\": container with ID starting with 43a18241480bf43e7de12d62727f363a0b56cd1b1064a639cd3528f56ef8ca4b not found: ID does not exist" containerID="43a18241480bf43e7de12d62727f363a0b56cd1b1064a639cd3528f56ef8ca4b"
Jul 30 10:50:41.494733 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.494669    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43a18241480bf43e7de12d62727f363a0b56cd1b1064a639cd3528f56ef8ca4b"} err="failed to get container status \"43a18241480bf43e7de12d62727f363a0b56cd1b1064a639cd3528f56ef8ca4b\": rpc error: code = NotFound desc = could not find container \"43a18241480bf43e7de12d62727f363a0b56cd1b1064a639cd3528f56ef8ca4b\": container with ID starting with 43a18241480bf43e7de12d62727f363a0b56cd1b1064a639cd3528f56ef8ca4b not found: ID does not exist"
Jul 30 10:50:41.494733 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.494692    2567 scope.go:117] "RemoveContainer" containerID="ad4798c55e928607dce242097f299f1dee4fe0a39a3ea5d36dde5b8f1b1f29cc"
Jul 30 10:50:41.503927 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.503882    2567 scope.go:117] "RemoveContainer" containerID="ad4798c55e928607dce242097f299f1dee4fe0a39a3ea5d36dde5b8f1b1f29cc"
Jul 30 10:50:41.504244 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:50:41.504205    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad4798c55e928607dce242097f299f1dee4fe0a39a3ea5d36dde5b8f1b1f29cc\": container with ID starting with ad4798c55e928607dce242097f299f1dee4fe0a39a3ea5d36dde5b8f1b1f29cc not found: ID does not exist" containerID="ad4798c55e928607dce242097f299f1dee4fe0a39a3ea5d36dde5b8f1b1f29cc"
Jul 30 10:50:41.504375 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.504242    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad4798c55e928607dce242097f299f1dee4fe0a39a3ea5d36dde5b8f1b1f29cc"} err="failed to get container status \"ad4798c55e928607dce242097f299f1dee4fe0a39a3ea5d36dde5b8f1b1f29cc\": rpc error: code = NotFound desc = could not find container \"ad4798c55e928607dce242097f299f1dee4fe0a39a3ea5d36dde5b8f1b1f29cc\": container with ID starting with ad4798c55e928607dce242097f299f1dee4fe0a39a3ea5d36dde5b8f1b1f29cc not found: ID does not exist"
Jul 30 10:50:41.505906 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.505884    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"]
Jul 30 10:50:41.510318 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.510298    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b75946/rate-test-model-b75946-kserve-5fdff645f5-8g47v"]
Jul 30 10:50:41.526262 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.526235    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-home\") pod \"9b282240-735f-4372-867c-ffeac9971cee\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") "
Jul 30 10:50:41.526356 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.526284    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-dshm\") pod \"9b282240-735f-4372-867c-ffeac9971cee\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") "
Jul 30 10:50:41.526356 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.526311    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-tmp-dir\") pod \"9b282240-735f-4372-867c-ffeac9971cee\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") "
Jul 30 10:50:41.526356 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.526330    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b282240-735f-4372-867c-ffeac9971cee-tls-certs\") pod \"9b282240-735f-4372-867c-ffeac9971cee\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") "
Jul 30 10:50:41.526493 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.526380    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlb8p\" (UniqueName: \"kubernetes.io/projected/9b282240-735f-4372-867c-ffeac9971cee-kube-api-access-qlb8p\") pod \"9b282240-735f-4372-867c-ffeac9971cee\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") "
Jul 30 10:50:41.526493 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.526411    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-model-cache\") pod \"9b282240-735f-4372-867c-ffeac9971cee\" (UID: \"9b282240-735f-4372-867c-ffeac9971cee\") "
Jul 30 10:50:41.526625 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.526563    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-home" (OuterVolumeSpecName: "home") pod "9b282240-735f-4372-867c-ffeac9971cee" (UID: "9b282240-735f-4372-867c-ffeac9971cee"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:50:41.526702 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.526651    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9b282240-735f-4372-867c-ffeac9971cee" (UID: "9b282240-735f-4372-867c-ffeac9971cee"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:50:41.526762 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.526677    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:50:41.526762 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.526730    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-model-cache" (OuterVolumeSpecName: "model-cache") pod "9b282240-735f-4372-867c-ffeac9971cee" (UID: "9b282240-735f-4372-867c-ffeac9971cee"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:50:41.528407 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.528382    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-dshm" (OuterVolumeSpecName: "dshm") pod "9b282240-735f-4372-867c-ffeac9971cee" (UID: "9b282240-735f-4372-867c-ffeac9971cee"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:50:41.528492 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.528426    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b282240-735f-4372-867c-ffeac9971cee-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9b282240-735f-4372-867c-ffeac9971cee" (UID: "9b282240-735f-4372-867c-ffeac9971cee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 10:50:41.528533 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.528507    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b282240-735f-4372-867c-ffeac9971cee-kube-api-access-qlb8p" (OuterVolumeSpecName: "kube-api-access-qlb8p") pod "9b282240-735f-4372-867c-ffeac9971cee" (UID: "9b282240-735f-4372-867c-ffeac9971cee"). InnerVolumeSpecName "kube-api-access-qlb8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:50:41.627182 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.627143    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:50:41.627182 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.627174    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:50:41.627182 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.627186    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b282240-735f-4372-867c-ffeac9971cee-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:50:41.627466 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.627195    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlb8p\" (UniqueName: \"kubernetes.io/projected/9b282240-735f-4372-867c-ffeac9971cee-kube-api-access-qlb8p\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:50:41.627466 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.627204    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b282240-735f-4372-867c-ffeac9971cee-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:50:41.807067 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.807036    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"]
Jul 30 10:50:41.808581 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:41.808559    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bdf164/rate-test-model-bdf164-kserve-8685d96855-x2tfr"]
Jul 30 10:50:42.062910 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.062835    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce" path="/var/lib/kubelet/pods/7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce/volumes"
Jul 30 10:50:42.063227 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.063214    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b282240-735f-4372-867c-ffeac9971cee" path="/var/lib/kubelet/pods/9b282240-735f-4372-867c-ffeac9971cee/volumes"
Jul 30 10:50:42.233584 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.233549    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7468-140667247360016"]
Jul 30 10:50:42.233879 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.233867    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce" containerName="main"
Jul 30 10:50:42.233939 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.233880    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce" containerName="main"
Jul 30 10:50:42.233939 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.233891    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b282240-735f-4372-867c-ffeac9971cee" containerName="main"
Jul 30 10:50:42.233939 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.233898    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b282240-735f-4372-867c-ffeac9971cee" containerName="main"
Jul 30 10:50:42.234035 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.233964    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b282240-735f-4372-867c-ffeac9971cee" containerName="main"
Jul 30 10:50:42.234035 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.233972    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a8826ae-1c2d-41e1-a0f1-8ae5627e6fce" containerName="main"
Jul 30 10:50:42.238645 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.238628    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7468-140667247360016"
Jul 30 10:50:42.241385 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.241361    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-v2xsn\""
Jul 30 10:50:42.243969 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.243944    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7468-140667247360016"]
Jul 30 10:50:42.332602 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.332520    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vpvd\" (UniqueName: \"kubernetes.io/projected/d5db6951-9d98-4796-b312-8011956716ec-kube-api-access-6vpvd\") pod \"test-curl-7468-140667247360016\" (UID: \"d5db6951-9d98-4796-b312-8011956716ec\") " pod="opendatahub/test-curl-7468-140667247360016"
Jul 30 10:50:42.433059 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.433026    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vpvd\" (UniqueName: \"kubernetes.io/projected/d5db6951-9d98-4796-b312-8011956716ec-kube-api-access-6vpvd\") pod \"test-curl-7468-140667247360016\" (UID: \"d5db6951-9d98-4796-b312-8011956716ec\") " pod="opendatahub/test-curl-7468-140667247360016"
Jul 30 10:50:42.442016 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.441987    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vpvd\" (UniqueName: \"kubernetes.io/projected/d5db6951-9d98-4796-b312-8011956716ec-kube-api-access-6vpvd\") pod \"test-curl-7468-140667247360016\" (UID: \"d5db6951-9d98-4796-b312-8011956716ec\") " pod="opendatahub/test-curl-7468-140667247360016"
Jul 30 10:50:42.549804 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.549772    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7468-140667247360016"
Jul 30 10:50:42.676208 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.676165    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7468-140667247360016"]
Jul 30 10:50:42.678468 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:50:42.678433    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5db6951_9d98_4796_b312_8011956716ec.slice/crio-29a6240eca3da3992f9b3ee0e3793cc100963565015efdde7977e247564e7939 WatchSource:0}: Error finding container 29a6240eca3da3992f9b3ee0e3793cc100963565015efdde7977e247564e7939: Status 404 returned error can't find the container with id 29a6240eca3da3992f9b3ee0e3793cc100963565015efdde7977e247564e7939
Jul 30 10:50:42.679733 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:42.679710    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 10:50:43.497841 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:43.497807    2567 generic.go:356] "Generic (PLEG): container finished" podID="d5db6951-9d98-4796-b312-8011956716ec" containerID="8053edc2877f01e47a8b84eca823594e248f45138729294061708d2651f3ee2e" exitCode=0
Jul 30 10:50:43.498008 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:43.497874    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7468-140667247360016" event={"ID":"d5db6951-9d98-4796-b312-8011956716ec","Type":"ContainerDied","Data":"8053edc2877f01e47a8b84eca823594e248f45138729294061708d2651f3ee2e"}
Jul 30 10:50:43.498008 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:43.497906    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7468-140667247360016" event={"ID":"d5db6951-9d98-4796-b312-8011956716ec","Type":"ContainerStarted","Data":"29a6240eca3da3992f9b3ee0e3793cc100963565015efdde7977e247564e7939"}
Jul 30 10:50:44.641756 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:44.641727    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7468-140667247360016"
Jul 30 10:50:44.654208 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:44.654185    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vpvd\" (UniqueName: \"kubernetes.io/projected/d5db6951-9d98-4796-b312-8011956716ec-kube-api-access-6vpvd\") pod \"d5db6951-9d98-4796-b312-8011956716ec\" (UID: \"d5db6951-9d98-4796-b312-8011956716ec\") "
Jul 30 10:50:44.656272 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:44.656246    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5db6951-9d98-4796-b312-8011956716ec-kube-api-access-6vpvd" (OuterVolumeSpecName: "kube-api-access-6vpvd") pod "d5db6951-9d98-4796-b312-8011956716ec" (UID: "d5db6951-9d98-4796-b312-8011956716ec"). InnerVolumeSpecName "kube-api-access-6vpvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:50:44.675593 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:44.675556    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7468-140667247360016_d5db6951-9d98-4796-b312-8011956716ec/test-curl-7468-140667247360016/0.log"
Jul 30 10:50:44.699429 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:44.699401    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7468-140667247360016"]
Jul 30 10:50:44.703143 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:44.703122    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7468-140667247360016"]
Jul 30 10:50:44.755103 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:44.755024    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vpvd\" (UniqueName: \"kubernetes.io/projected/d5db6951-9d98-4796-b312-8011956716ec-kube-api-access-6vpvd\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:50:45.509151 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.509121    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29a6240eca3da3992f9b3ee0e3793cc100963565015efdde7977e247564e7939"
Jul 30 10:50:45.509151 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.509141    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7468-140667247360016"
Jul 30 10:50:45.519695 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:50:45.519660    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7468-140667247360016\" is forbidden: User \"system:node:ip-10-0-128-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-146.ec2.internal' and this object" podUID="d5db6951-9d98-4796-b312-8011956716ec" pod="opendatahub/test-curl-7468-140667247360016"
Jul 30 10:50:45.550408 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.550370    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"]
Jul 30 10:50:45.550882 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.550866    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5db6951-9d98-4796-b312-8011956716ec" containerName="test-curl-7468-140667247360016"
Jul 30 10:50:45.550961 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.550885    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5db6951-9d98-4796-b312-8011956716ec" containerName="test-curl-7468-140667247360016"
Jul 30 10:50:45.551005 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.550994    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5db6951-9d98-4796-b312-8011956716ec" containerName="test-curl-7468-140667247360016"
Jul 30 10:50:45.554235 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.554213    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.556769 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.556702    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-78eb1c-openshift-default-dockercfg-c9qj4\""
Jul 30 10:50:45.556769 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.556746    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 10:50:45.561926 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.561523    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-credential-socket\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.561926 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.561561    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr755\" (UniqueName: \"kubernetes.io/projected/a395fd5f-df04-4b52-94d7-140cf7741d5d-kube-api-access-pr755\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.561926 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.561610    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-token\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.561926 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.561633    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a395fd5f-df04-4b52-94d7-140cf7741d5d-istiod-ca-cert\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.561926 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.561665    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-data\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.561926 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.561691    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-workload-certs\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.561926 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.561721    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-envoy\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.561926 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.561757    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-workload-socket\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.561926 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.561794    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-podinfo\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.564078 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.564045    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"]
Jul 30 10:50:45.577372 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:50:45.577310    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7468-140667247360016\" is forbidden: User \"system:node:ip-10-0-128-146.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-146.ec2.internal' and this object" podUID="d5db6951-9d98-4796-b312-8011956716ec" pod="opendatahub/test-curl-7468-140667247360016"
Jul 30 10:50:45.662636 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.662598    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-credential-socket\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.663082 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.662645    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr755\" (UniqueName: \"kubernetes.io/projected/a395fd5f-df04-4b52-94d7-140cf7741d5d-kube-api-access-pr755\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.663082 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.662689    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-token\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.663082 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.662712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a395fd5f-df04-4b52-94d7-140cf7741d5d-istiod-ca-cert\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.663082 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.662853    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-data\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.663082 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.662897    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-workload-certs\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.663082 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.662988    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-credential-socket\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.663082 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.663012    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-envoy\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.663082 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.663054    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-workload-socket\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.663526 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.663097    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-podinfo\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.663526 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.663188    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-data\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.663526 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.663278    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-workload-certs\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.663526 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.663409    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a395fd5f-df04-4b52-94d7-140cf7741d5d-istiod-ca-cert\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.663526 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.663412    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-workload-socket\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.665186 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.665159    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-envoy\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.665446 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.665430    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-podinfo\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.671294 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.671277    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr755\" (UniqueName: \"kubernetes.io/projected/a395fd5f-df04-4b52-94d7-140cf7741d5d-kube-api-access-pr755\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.671401 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.671360    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-token\") pod \"e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") " pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:45.865880 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:45.865789    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:46.004217 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:46.004192    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"]
Jul 30 10:50:46.005824 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:50:46.005797    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda395fd5f_df04_4b52_94d7_140cf7741d5d.slice/crio-0197fe100afef686623182c95b6c2627b8938a194c920d87258435111ddcc4fc WatchSource:0}: Error finding container 0197fe100afef686623182c95b6c2627b8938a194c920d87258435111ddcc4fc: Status 404 returned error can't find the container with id 0197fe100afef686623182c95b6c2627b8938a194c920d87258435111ddcc4fc
Jul 30 10:50:46.064385 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:46.064307    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5db6951-9d98-4796-b312-8011956716ec" path="/var/lib/kubelet/pods/d5db6951-9d98-4796-b312-8011956716ec/volumes"
Jul 30 10:50:46.515684 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:46.515648    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl" event={"ID":"a395fd5f-df04-4b52-94d7-140cf7741d5d","Type":"ContainerStarted","Data":"0197fe100afef686623182c95b6c2627b8938a194c920d87258435111ddcc4fc"}
Jul 30 10:50:49.231966 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:49.231929    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 10:50:49.232243 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:49.232000    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 10:50:49.232243 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:49.232031    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 10:50:49.532178 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:49.532089    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl" event={"ID":"a395fd5f-df04-4b52-94d7-140cf7741d5d","Type":"ContainerStarted","Data":"e6c5b77775952a52b31ac96db0ccbb1e8f13fb6f3fc071940bca9da3f8c48862"}
Jul 30 10:50:49.555935 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:49.555886    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl" podStartSLOduration=1.3322697159999999 podStartE2EDuration="4.555867147s" totalImagesPullingTime="3.223597431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:50:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:50:49.554686424 +0000 UTC m=+3686.048881943" watchObservedRunningTime="2026-07-30 10:50:49.555867147 +0000 UTC m=+3686.050062653"
Jul 30 10:50:49.866669 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:49.866587    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:50.871316 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:50.871287    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:51.541596 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:51.541567    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:51.542363 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:51.542326    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:50:57.692666 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:50:57.692630    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:51:06.543441 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:06.543401    2567 patch_prober.go:28] interesting pod/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 30 10:51:06.543873 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:06.543458    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl" podUID="a395fd5f-df04-4b52-94d7-140cf7741d5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 30 10:51:06.615297 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:06.615265    2567 generic.go:356] "Generic (PLEG): container finished" podID="a395fd5f-df04-4b52-94d7-140cf7741d5d" containerID="e6c5b77775952a52b31ac96db0ccbb1e8f13fb6f3fc071940bca9da3f8c48862" exitCode=0
Jul 30 10:51:06.615475 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:06.615306    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl" event={"ID":"a395fd5f-df04-4b52-94d7-140cf7741d5d","Type":"ContainerDied","Data":"e6c5b77775952a52b31ac96db0ccbb1e8f13fb6f3fc071940bca9da3f8c48862"}
Jul 30 10:51:06.615685 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:06.615670    2567 scope.go:117] "RemoveContainer" containerID="e6c5b77775952a52b31ac96db0ccbb1e8f13fb6f3fc071940bca9da3f8c48862"
Jul 30 10:51:06.616739 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:06.616709    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 10:51:06.616805 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:06.616776    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 10:51:06.616847 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:06.616806    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 10:51:07.621840 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:07.621804    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl" event={"ID":"a395fd5f-df04-4b52-94d7-140cf7741d5d","Type":"ContainerStarted","Data":"91a9a712697c505d140d496b5087e05b2322ea8817fb8434052f689c3a376134"}
Jul 30 10:51:07.866995 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:07.866946    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:51:07.867212 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:07.867025    2567 patch_prober.go:28] interesting pod/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 30 10:51:07.867212 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:07.867094    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl" podUID="a395fd5f-df04-4b52-94d7-140cf7741d5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 30 10:51:08.866477 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:08.866433    2567 patch_prober.go:28] interesting pod/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 30 10:51:08.866987 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:08.866511    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl" podUID="a395fd5f-df04-4b52-94d7-140cf7741d5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 30 10:51:09.867265 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:09.867218    2567 patch_prober.go:28] interesting pod/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 30 10:51:09.867811 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:09.867289    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl" podUID="a395fd5f-df04-4b52-94d7-140cf7741d5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 30 10:51:10.867190 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:10.867150    2567 patch_prober.go:28] interesting pod/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 30 10:51:10.867531 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:10.867499    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl" podUID="a395fd5f-df04-4b52-94d7-140cf7741d5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 30 10:51:11.870091 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:11.870062    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:51:11.870505 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:11.870319    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:51:11.871070 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:11.871053    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:51:17.553967 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.553927    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"]
Jul 30 10:51:17.561463 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.561441    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.564324 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.564307    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-2aadf4-openshift-default-dockercfg-rmpqv\""
Jul 30 10:51:17.569524 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.569499    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"]
Jul 30 10:51:17.636153 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.636112    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-workload-certs\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.636153 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.636155    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-workload-socket\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.636434 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.636176    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-token\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.636434 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.636223    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istiod-ca-cert\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.636434 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.636255    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-podinfo\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.636434 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.636284    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-credential-socket\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.636434 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.636322    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-data\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.636434 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.636386    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px6f8\" (UniqueName: \"kubernetes.io/projected/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-kube-api-access-px6f8\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.636434 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.636411    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-envoy\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.737166 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.737127    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-data\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.737166 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.737168    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px6f8\" (UniqueName: \"kubernetes.io/projected/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-kube-api-access-px6f8\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.737481 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.737190    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-envoy\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.737481 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.737230    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-workload-certs\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.737481 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.737255    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-workload-socket\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.737481 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.737282    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-token\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.737481 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.737317    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istiod-ca-cert\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.737481 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.737389    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-podinfo\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.737481 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.737430    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-credential-socket\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.737855 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.737759    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-data\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.737942 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.737919    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-workload-certs\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.738000 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.737973    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-credential-socket\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.738061 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.738037    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-workload-socket\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.738146 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.738128    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istiod-ca-cert\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.739694 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.739666    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-envoy\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.740033 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.740010    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-podinfo\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.745742 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.745716    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-token\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.746258 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.746238    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px6f8\" (UniqueName: \"kubernetes.io/projected/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-kube-api-access-px6f8\") pod \"e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") " pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:17.874122 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:17.874092    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:18.011709 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:18.011686    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"]
Jul 30 10:51:18.013974 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:51:18.013944    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d0089aa_61c9_4d6e_bf75_44ebe7a29b08.slice/crio-2bc8eda71daade090e6f682a956177c15dfe144a1fd8bd8da2ec0b0a1e9b6848 WatchSource:0}: Error finding container 2bc8eda71daade090e6f682a956177c15dfe144a1fd8bd8da2ec0b0a1e9b6848: Status 404 returned error can't find the container with id 2bc8eda71daade090e6f682a956177c15dfe144a1fd8bd8da2ec0b0a1e9b6848
Jul 30 10:51:18.016481 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:18.016448    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 10:51:18.016562 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:18.016518    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 10:51:18.016562 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:18.016546    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 10:51:18.674053 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:18.674018    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8" event={"ID":"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08","Type":"ContainerStarted","Data":"d82ef21683bec72aaaae30b3a87c78908f01dfdc0951e73bfb8130a1c0cc8607"}
Jul 30 10:51:18.674494 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:18.674079    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8" event={"ID":"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08","Type":"ContainerStarted","Data":"2bc8eda71daade090e6f682a956177c15dfe144a1fd8bd8da2ec0b0a1e9b6848"}
Jul 30 10:51:18.698101 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:18.698050    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8" podStartSLOduration=1.698033737 podStartE2EDuration="1.698033737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:51:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:51:18.69675929 +0000 UTC m=+3715.190954807" watchObservedRunningTime="2026-07-30 10:51:18.698033737 +0000 UTC m=+3715.192229322"
Jul 30 10:51:18.874493 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:18.874457    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:19.879266 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:19.879237    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:20.683201 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:20.683169    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:20.684205 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:20.684184    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:31.110639 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:31.110606    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:51:39.779176 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:39.779143    2567 generic.go:356] "Generic (PLEG): container finished" podID="8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" containerID="d82ef21683bec72aaaae30b3a87c78908f01dfdc0951e73bfb8130a1c0cc8607" exitCode=0
Jul 30 10:51:39.779635 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:39.779209    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8" event={"ID":"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08","Type":"ContainerDied","Data":"d82ef21683bec72aaaae30b3a87c78908f01dfdc0951e73bfb8130a1c0cc8607"}
Jul 30 10:51:39.779695 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:39.779649    2567 scope.go:117] "RemoveContainer" containerID="d82ef21683bec72aaaae30b3a87c78908f01dfdc0951e73bfb8130a1c0cc8607"
Jul 30 10:51:39.780806 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:39.780777    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 10:51:39.780878 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:39.780843    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 10:51:39.780878 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:39.780874    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 30 10:51:40.786387 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:40.786330    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8" event={"ID":"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08","Type":"ContainerStarted","Data":"3573954d736e2e6d70403ead9392a62b940b23e774685fbbc181bc16f1109260"}
Jul 30 10:51:40.875417 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:40.875372    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:40.875642 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:40.875510    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 30 10:51:40.875642 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:40.875552    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8" podUID="8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 30 10:51:41.875088 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:41.875048    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 30 10:51:41.875721 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:41.875683    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8" podUID="8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 30 10:51:42.875013 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:42.874968    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 30 10:51:42.875316 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:42.875284    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8" podUID="8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 30 10:51:43.875647 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:43.875607    2567 patch_prober.go:28] interesting pod/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 30 10:51:43.876287 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:43.876239    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8" podUID="8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 30 10:51:44.878382 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:44.878351    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:44.878886 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:44.878768    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:51:44.879751 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:51:44.879731    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:52:05.716011 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.715977    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"]
Jul 30 10:52:05.719790 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.719768    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.722719 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.722701    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-78eb1c\"/\"default-dockercfg-9ffgj\""
Jul 30 10:52:05.723750 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.723730    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-78eb1c\"/\"kube-root-ca.crt\""
Jul 30 10:52:05.723864 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.723760    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-78eb1c\"/\"openshift-service-ca.crt\""
Jul 30 10:52:05.723864 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.723816    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-78eb1c\"/\"ipp-route-78eb1c-kserve-self-signed-certs\""
Jul 30 10:52:05.729663 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.729643    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"]
Jul 30 10:52:05.771923 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.771891    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-tls-certs\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.772095 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.771927    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-home\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.772095 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.771959    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-dshm\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.772095 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.771983    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l59ts\" (UniqueName: \"kubernetes.io/projected/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-kube-api-access-l59ts\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.772095 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.772008    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-model-cache\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.772095 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.772036    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-tmp-dir\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.873297 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.873254    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-tls-certs\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.873520 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.873302    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-home\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.873520 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.873374    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-dshm\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.873520 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.873400    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l59ts\" (UniqueName: \"kubernetes.io/projected/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-kube-api-access-l59ts\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.873520 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.873426    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-model-cache\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.873520 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.873466    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-tmp-dir\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.873872 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.873850    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-home\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.873939 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.873871    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-model-cache\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.873939 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.873918    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-tmp-dir\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.876171 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.876152    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-dshm\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.876372 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.876357    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-tls-certs\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:05.882258 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:05.882239    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l59ts\" (UniqueName: \"kubernetes.io/projected/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-kube-api-access-l59ts\") pod \"ipp-route-78eb1c-kserve-7779c474c5-gmj8h\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") " pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:06.030326 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:06.030234    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:06.162399 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:06.162374    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"]
Jul 30 10:52:06.164150 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:52:06.164125    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0ec7a57_9905_4cbd_b951_fa63884c4e8a.slice/crio-74a2ab3a89f2c7ce1d90c8cf22b460f2661ab1d9dee1e96af9f6329a10f83080 WatchSource:0}: Error finding container 74a2ab3a89f2c7ce1d90c8cf22b460f2661ab1d9dee1e96af9f6329a10f83080: Status 404 returned error can't find the container with id 74a2ab3a89f2c7ce1d90c8cf22b460f2661ab1d9dee1e96af9f6329a10f83080
Jul 30 10:52:06.910015 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:06.909981    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h" event={"ID":"e0ec7a57-9905-4cbd-b951-fa63884c4e8a","Type":"ContainerStarted","Data":"25c39dceb45472d0db78a2562004ec9379eb76b1446f3e052187c2222e153204"}
Jul 30 10:52:06.910015 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:06.910022    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h" event={"ID":"e0ec7a57-9905-4cbd-b951-fa63884c4e8a","Type":"ContainerStarted","Data":"74a2ab3a89f2c7ce1d90c8cf22b460f2661ab1d9dee1e96af9f6329a10f83080"}
Jul 30 10:52:06.927856 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:06.927803    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h" podStartSLOduration=1.927787446 podStartE2EDuration="1.927787446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:52:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:52:06.926358279 +0000 UTC m=+3763.420553776" watchObservedRunningTime="2026-07-30 10:52:06.927787446 +0000 UTC m=+3763.421982951"
Jul 30 10:52:09.804205 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:09.804167    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:52:16.031426 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:16.031300    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:16.031426 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:16.031379    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:16.044495 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:16.044459    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:16.967849 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:16.967817    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:52:43.090061 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:52:43.090026    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:53:01.097039 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:01.097003    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:53:22.687087 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:22.687053    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx"]
Jul 30 10:53:22.690392 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:22.690374    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx"
Jul 30 10:53:22.692853 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:22.692828    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 10:53:22.692957 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:22.692866    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j6dj6\""
Jul 30 10:53:22.699199 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:22.699174    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx"]
Jul 30 10:53:22.761982 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:22.761937    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/291e5f85-d882-4b8d-a0c1-c4515ad51ee1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx\" (UID: \"291e5f85-d882-4b8d-a0c1-c4515ad51ee1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx"
Jul 30 10:53:22.862685 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:22.862630    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/291e5f85-d882-4b8d-a0c1-c4515ad51ee1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx\" (UID: \"291e5f85-d882-4b8d-a0c1-c4515ad51ee1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx"
Jul 30 10:53:22.863278 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:22.863256    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/291e5f85-d882-4b8d-a0c1-c4515ad51ee1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx\" (UID: \"291e5f85-d882-4b8d-a0c1-c4515ad51ee1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx"
Jul 30 10:53:23.001499 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:23.001407    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx"
Jul 30 10:53:23.131641 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:23.131613    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx"]
Jul 30 10:53:23.133927 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:53:23.133896    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291e5f85_d882_4b8d_a0c1_c4515ad51ee1.slice/crio-7936857f3ae740829f61ed82e22028c18056ed20c02df54b4b0b8e5e2466e6c8 WatchSource:0}: Error finding container 7936857f3ae740829f61ed82e22028c18056ed20c02df54b4b0b8e5e2466e6c8: Status 404 returned error can't find the container with id 7936857f3ae740829f61ed82e22028c18056ed20c02df54b4b0b8e5e2466e6c8
Jul 30 10:53:23.268410 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:23.268308    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx" event={"ID":"291e5f85-d882-4b8d-a0c1-c4515ad51ee1","Type":"ContainerStarted","Data":"7936857f3ae740829f61ed82e22028c18056ed20c02df54b4b0b8e5e2466e6c8"}
Jul 30 10:53:24.274849 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:24.274812    2567 generic.go:356] "Generic (PLEG): container finished" podID="291e5f85-d882-4b8d-a0c1-c4515ad51ee1" containerID="56797d97080b9d2a2c3af330203a641c35499419ba53daacf3ca6eb5cb423eac" exitCode=0
Jul 30 10:53:24.275226 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:24.274884    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx" event={"ID":"291e5f85-d882-4b8d-a0c1-c4515ad51ee1","Type":"ContainerDied","Data":"56797d97080b9d2a2c3af330203a641c35499419ba53daacf3ca6eb5cb423eac"}
Jul 30 10:53:25.411852 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:25.411828    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx"
Jul 30 10:53:25.482199 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:25.482157    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/291e5f85-d882-4b8d-a0c1-c4515ad51ee1-maas-api-service-ca\") pod \"291e5f85-d882-4b8d-a0c1-c4515ad51ee1\" (UID: \"291e5f85-d882-4b8d-a0c1-c4515ad51ee1\") "
Jul 30 10:53:25.482497 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:25.482471    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/291e5f85-d882-4b8d-a0c1-c4515ad51ee1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "291e5f85-d882-4b8d-a0c1-c4515ad51ee1" (UID: "291e5f85-d882-4b8d-a0c1-c4515ad51ee1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 10:53:25.583262 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:25.583181    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/291e5f85-d882-4b8d-a0c1-c4515ad51ee1-maas-api-service-ca\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:53:26.285539 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:26.285504    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx" event={"ID":"291e5f85-d882-4b8d-a0c1-c4515ad51ee1","Type":"ContainerDied","Data":"7936857f3ae740829f61ed82e22028c18056ed20c02df54b4b0b8e5e2466e6c8"}
Jul 30 10:53:26.285539 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:26.285524    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx"
Jul 30 10:53:26.285742 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:26.285559    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7936857f3ae740829f61ed82e22028c18056ed20c02df54b4b0b8e5e2466e6c8"
Jul 30 10:53:28.753064 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:28.753028    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx"]
Jul 30 10:53:28.755920 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:28.755890    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8cd2f5-rnqhx"]
Jul 30 10:53:30.063355 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:30.063313    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="291e5f85-d882-4b8d-a0c1-c4515ad51ee1" path="/var/lib/kubelet/pods/291e5f85-d882-4b8d-a0c1-c4515ad51ee1/volumes"
Jul 30 10:53:32.997812 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:32.997773    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:53:39.180409 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:39.180371    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n"]
Jul 30 10:53:39.180804 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:39.180716    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="291e5f85-d882-4b8d-a0c1-c4515ad51ee1" containerName="revoke-keys"
Jul 30 10:53:39.180804 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:39.180726    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="291e5f85-d882-4b8d-a0c1-c4515ad51ee1" containerName="revoke-keys"
Jul 30 10:53:39.180804 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:39.180798    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="291e5f85-d882-4b8d-a0c1-c4515ad51ee1" containerName="revoke-keys"
Jul 30 10:53:39.184892 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:39.184873    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n"
Jul 30 10:53:39.187580 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:39.187556    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 10:53:39.187849 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:39.187835    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j6dj6\""
Jul 30 10:53:39.192442 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:39.192095    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n"]
Jul 30 10:53:39.210297 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:39.210269    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd0f618-15bc-4d33-9c4c-bc68ba2ad766-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n\" (UID: \"9bd0f618-15bc-4d33-9c4c-bc68ba2ad766\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n"
Jul 30 10:53:39.311623 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:39.311586    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd0f618-15bc-4d33-9c4c-bc68ba2ad766-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n\" (UID: \"9bd0f618-15bc-4d33-9c4c-bc68ba2ad766\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n"
Jul 30 10:53:39.312235 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:39.312215    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd0f618-15bc-4d33-9c4c-bc68ba2ad766-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n\" (UID: \"9bd0f618-15bc-4d33-9c4c-bc68ba2ad766\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n"
Jul 30 10:53:39.496757 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:39.496668    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n"
Jul 30 10:53:39.839281 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:39.839252    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n"]
Jul 30 10:53:39.841220 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:53:39.841186    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bd0f618_15bc_4d33_9c4c_bc68ba2ad766.slice/crio-78961ca418f09c02ade39941cd582ca30dc26a33cd88926dc1e6abccb7a3a4ea WatchSource:0}: Error finding container 78961ca418f09c02ade39941cd582ca30dc26a33cd88926dc1e6abccb7a3a4ea: Status 404 returned error can't find the container with id 78961ca418f09c02ade39941cd582ca30dc26a33cd88926dc1e6abccb7a3a4ea
Jul 30 10:53:40.354474 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:40.354434    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n" event={"ID":"9bd0f618-15bc-4d33-9c4c-bc68ba2ad766","Type":"ContainerStarted","Data":"e7b37ffb3848afa95a05c9e5dea71445994b8d37c7d802df8f53a20b207cc587"}
Jul 30 10:53:40.354874 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:40.354493    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n" event={"ID":"9bd0f618-15bc-4d33-9c4c-bc68ba2ad766","Type":"ContainerStarted","Data":"78961ca418f09c02ade39941cd582ca30dc26a33cd88926dc1e6abccb7a3a4ea"}
Jul 30 10:53:40.375176 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:40.373573    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n" podStartSLOduration=1.373550668 podStartE2EDuration="1.373550668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:53:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:53:40.368816382 +0000 UTC m=+3856.863011887" watchObservedRunningTime="2026-07-30 10:53:40.373550668 +0000 UTC m=+3856.867746173"
Jul 30 10:53:41.361067 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:41.361031    2567 generic.go:356] "Generic (PLEG): container finished" podID="9bd0f618-15bc-4d33-9c4c-bc68ba2ad766" containerID="e7b37ffb3848afa95a05c9e5dea71445994b8d37c7d802df8f53a20b207cc587" exitCode=0
Jul 30 10:53:41.361538 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:41.361111    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n" event={"ID":"9bd0f618-15bc-4d33-9c4c-bc68ba2ad766","Type":"ContainerDied","Data":"e7b37ffb3848afa95a05c9e5dea71445994b8d37c7d802df8f53a20b207cc587"}
Jul 30 10:53:42.498718 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:42.498691    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n"
Jul 30 10:53:42.536102 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:42.536067    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd0f618-15bc-4d33-9c4c-bc68ba2ad766-maas-api-service-ca\") pod \"9bd0f618-15bc-4d33-9c4c-bc68ba2ad766\" (UID: \"9bd0f618-15bc-4d33-9c4c-bc68ba2ad766\") "
Jul 30 10:53:42.536546 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:42.536518    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bd0f618-15bc-4d33-9c4c-bc68ba2ad766-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9bd0f618-15bc-4d33-9c4c-bc68ba2ad766" (UID: "9bd0f618-15bc-4d33-9c4c-bc68ba2ad766"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 10:53:42.637525 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:42.637495    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bd0f618-15bc-4d33-9c4c-bc68ba2ad766-maas-api-service-ca\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:53:43.372045 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:43.372005    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n"
Jul 30 10:53:43.372227 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:43.372042    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n" event={"ID":"9bd0f618-15bc-4d33-9c4c-bc68ba2ad766","Type":"ContainerDied","Data":"78961ca418f09c02ade39941cd582ca30dc26a33cd88926dc1e6abccb7a3a4ea"}
Jul 30 10:53:43.372227 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:43.372091    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78961ca418f09c02ade39941cd582ca30dc26a33cd88926dc1e6abccb7a3a4ea"
Jul 30 10:53:53.899172 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:53.899088    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:53:55.144437 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:55.144400    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n"]
Jul 30 10:53:55.148401 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:55.148372    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-78eb1c-d4s5n"]
Jul 30 10:53:56.064488 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:53:56.064457    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bd0f618-15bc-4d33-9c4c-bc68ba2ad766" path="/var/lib/kubelet/pods/9bd0f618-15bc-4d33-9c4c-bc68ba2ad766/volumes"
Jul 30 10:54:00.505316 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:00.505282    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"]
Jul 30 10:54:04.605027 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:04.604996    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:54:15.524849 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.524799    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h" podUID="e0ec7a57-9905-4cbd-b951-fa63884c4e8a" containerName="main" containerID="cri-o://25c39dceb45472d0db78a2562004ec9379eb76b1446f3e052187c2222e153204" gracePeriod=45
Jul 30 10:54:15.776202 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.776139    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:54:15.928364 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.928310    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-model-cache\") pod \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") "
Jul 30 10:54:15.928602 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.928401    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-home\") pod \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") "
Jul 30 10:54:15.928602 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.928446    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-dshm\") pod \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") "
Jul 30 10:54:15.928602 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.928496    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l59ts\" (UniqueName: \"kubernetes.io/projected/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-kube-api-access-l59ts\") pod \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") "
Jul 30 10:54:15.928602 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.928527    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-tls-certs\") pod \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") "
Jul 30 10:54:15.928602 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.928600    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-tmp-dir\") pod \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\" (UID: \"e0ec7a57-9905-4cbd-b951-fa63884c4e8a\") "
Jul 30 10:54:15.928881 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.928633    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-model-cache" (OuterVolumeSpecName: "model-cache") pod "e0ec7a57-9905-4cbd-b951-fa63884c4e8a" (UID: "e0ec7a57-9905-4cbd-b951-fa63884c4e8a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:54:15.928881 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.928665    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-home" (OuterVolumeSpecName: "home") pod "e0ec7a57-9905-4cbd-b951-fa63884c4e8a" (UID: "e0ec7a57-9905-4cbd-b951-fa63884c4e8a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:54:15.928881 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.928841    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e0ec7a57-9905-4cbd-b951-fa63884c4e8a" (UID: "e0ec7a57-9905-4cbd-b951-fa63884c4e8a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:54:15.928992 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.928917    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-model-cache\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:15.928992 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.928935    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-home\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:15.930797 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.930766    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e0ec7a57-9905-4cbd-b951-fa63884c4e8a" (UID: "e0ec7a57-9905-4cbd-b951-fa63884c4e8a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 10:54:15.930915 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.930797    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-dshm" (OuterVolumeSpecName: "dshm") pod "e0ec7a57-9905-4cbd-b951-fa63884c4e8a" (UID: "e0ec7a57-9905-4cbd-b951-fa63884c4e8a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:54:15.930915 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:15.930820    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-kube-api-access-l59ts" (OuterVolumeSpecName: "kube-api-access-l59ts") pod "e0ec7a57-9905-4cbd-b951-fa63884c4e8a" (UID: "e0ec7a57-9905-4cbd-b951-fa63884c4e8a"). InnerVolumeSpecName "kube-api-access-l59ts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:54:16.030145 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:16.030045    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-dshm\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:16.030145 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:16.030087    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l59ts\" (UniqueName: \"kubernetes.io/projected/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-kube-api-access-l59ts\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:16.030145 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:16.030097    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-tls-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:16.030145 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:16.030107    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ec7a57-9905-4cbd-b951-fa63884c4e8a-tmp-dir\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:16.538751 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:16.538710    2567 generic.go:356] "Generic (PLEG): container finished" podID="e0ec7a57-9905-4cbd-b951-fa63884c4e8a" containerID="25c39dceb45472d0db78a2562004ec9379eb76b1446f3e052187c2222e153204" exitCode=0
Jul 30 10:54:16.539181 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:16.538758    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"
Jul 30 10:54:16.539181 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:16.538757    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h" event={"ID":"e0ec7a57-9905-4cbd-b951-fa63884c4e8a","Type":"ContainerDied","Data":"25c39dceb45472d0db78a2562004ec9379eb76b1446f3e052187c2222e153204"}
Jul 30 10:54:16.539181 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:16.538799    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h" event={"ID":"e0ec7a57-9905-4cbd-b951-fa63884c4e8a","Type":"ContainerDied","Data":"74a2ab3a89f2c7ce1d90c8cf22b460f2661ab1d9dee1e96af9f6329a10f83080"}
Jul 30 10:54:16.539181 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:16.538820    2567 scope.go:117] "RemoveContainer" containerID="25c39dceb45472d0db78a2562004ec9379eb76b1446f3e052187c2222e153204"
Jul 30 10:54:16.547503 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:16.547485    2567 scope.go:117] "RemoveContainer" containerID="25c39dceb45472d0db78a2562004ec9379eb76b1446f3e052187c2222e153204"
Jul 30 10:54:16.547740 ip-10-0-128-146 kubenswrapper[2567]: E0730 10:54:16.547719    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25c39dceb45472d0db78a2562004ec9379eb76b1446f3e052187c2222e153204\": container with ID starting with 25c39dceb45472d0db78a2562004ec9379eb76b1446f3e052187c2222e153204 not found: ID does not exist" containerID="25c39dceb45472d0db78a2562004ec9379eb76b1446f3e052187c2222e153204"
Jul 30 10:54:16.547794 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:16.547749    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25c39dceb45472d0db78a2562004ec9379eb76b1446f3e052187c2222e153204"} err="failed to get container status \"25c39dceb45472d0db78a2562004ec9379eb76b1446f3e052187c2222e153204\": rpc error: code = NotFound desc = could not find container \"25c39dceb45472d0db78a2562004ec9379eb76b1446f3e052187c2222e153204\": container with ID starting with 25c39dceb45472d0db78a2562004ec9379eb76b1446f3e052187c2222e153204 not found: ID does not exist"
Jul 30 10:54:16.557418 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:16.557389    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"]
Jul 30 10:54:16.561105 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:16.561083    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-78eb1c/ipp-route-78eb1c-kserve-7779c474c5-gmj8h"]
Jul 30 10:54:18.063053 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:18.063020    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0ec7a57-9905-4cbd-b951-fa63884c4e8a" path="/var/lib/kubelet/pods/e0ec7a57-9905-4cbd-b951-fa63884c4e8a/volumes"
Jul 30 10:54:24.190738 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:24.190708    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:54:24.197603 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:24.197572    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:54:24.204739 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:24.204718    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:54:24.210440 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:24.210420    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wf4hv_bafeef50-65d3-4fe3-8c83-c70a40f46d73/ovn-acl-logging/0.log"
Jul 30 10:54:43.477892 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:43.477858    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"]
Jul 30 10:54:43.478404 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:43.478087    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl" podUID="a395fd5f-df04-4b52-94d7-140cf7741d5d" containerName="istio-proxy" containerID="cri-o://91a9a712697c505d140d496b5087e05b2322ea8817fb8434052f689c3a376134" gracePeriod=30
Jul 30 10:54:43.867280 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:43.867177    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj"]
Jul 30 10:54:43.867651 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:43.867632    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0ec7a57-9905-4cbd-b951-fa63884c4e8a" containerName="main"
Jul 30 10:54:43.867740 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:43.867654    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0ec7a57-9905-4cbd-b951-fa63884c4e8a" containerName="main"
Jul 30 10:54:43.867740 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:43.867666    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bd0f618-15bc-4d33-9c4c-bc68ba2ad766" containerName="revoke-keys"
Jul 30 10:54:43.867740 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:43.867675    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bd0f618-15bc-4d33-9c4c-bc68ba2ad766" containerName="revoke-keys"
Jul 30 10:54:43.867887 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:43.867771    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bd0f618-15bc-4d33-9c4c-bc68ba2ad766" containerName="revoke-keys"
Jul 30 10:54:43.867887 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:43.867792    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0ec7a57-9905-4cbd-b951-fa63884c4e8a" containerName="main"
Jul 30 10:54:43.870850 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:43.870831    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj"
Jul 30 10:54:43.873723 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:43.873707    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j6dj6\""
Jul 30 10:54:43.874043 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:43.874027    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 10:54:43.879113 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:43.879090    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj"]
Jul 30 10:54:43.960528 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:43.960492    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a245a488-26a4-4475-8bd0-d7e6d6d1f397-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj\" (UID: \"a245a488-26a4-4475-8bd0-d7e6d6d1f397\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj"
Jul 30 10:54:44.060947 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:44.060914    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a245a488-26a4-4475-8bd0-d7e6d6d1f397-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj\" (UID: \"a245a488-26a4-4475-8bd0-d7e6d6d1f397\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj"
Jul 30 10:54:44.061554 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:44.061535    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a245a488-26a4-4475-8bd0-d7e6d6d1f397-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj\" (UID: \"a245a488-26a4-4475-8bd0-d7e6d6d1f397\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj"
Jul 30 10:54:44.182322 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:44.182287    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj"
Jul 30 10:54:44.315897 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:44.315873    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj"]
Jul 30 10:54:44.317630 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:54:44.317603    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda245a488_26a4_4475_8bd0_d7e6d6d1f397.slice/crio-3db0e586679199c4073e54677ed3115bdc4d1addb69f64dc2c99961be5ef0b02 WatchSource:0}: Error finding container 3db0e586679199c4073e54677ed3115bdc4d1addb69f64dc2c99961be5ef0b02: Status 404 returned error can't find the container with id 3db0e586679199c4073e54677ed3115bdc4d1addb69f64dc2c99961be5ef0b02
Jul 30 10:54:44.676553 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:44.676523    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj" event={"ID":"a245a488-26a4-4475-8bd0-d7e6d6d1f397","Type":"ContainerStarted","Data":"056ff9345fb41e73d14b3bd7da86023de8ae7b17623742add60f6c5a5929ff87"}
Jul 30 10:54:44.676943 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:44.676575    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj" event={"ID":"a245a488-26a4-4475-8bd0-d7e6d6d1f397","Type":"ContainerStarted","Data":"3db0e586679199c4073e54677ed3115bdc4d1addb69f64dc2c99961be5ef0b02"}
Jul 30 10:54:44.693094 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:44.693051    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj" podStartSLOduration=1.693035461 podStartE2EDuration="1.693035461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:54:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:54:44.691114797 +0000 UTC m=+3921.185310304" watchObservedRunningTime="2026-07-30 10:54:44.693035461 +0000 UTC m=+3921.187230944"
Jul 30 10:54:45.682220 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:45.682184    2567 generic.go:356] "Generic (PLEG): container finished" podID="a245a488-26a4-4475-8bd0-d7e6d6d1f397" containerID="056ff9345fb41e73d14b3bd7da86023de8ae7b17623742add60f6c5a5929ff87" exitCode=0
Jul 30 10:54:45.682645 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:45.682264    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj" event={"ID":"a245a488-26a4-4475-8bd0-d7e6d6d1f397","Type":"ContainerDied","Data":"056ff9345fb41e73d14b3bd7da86023de8ae7b17623742add60f6c5a5929ff87"}
Jul 30 10:54:46.820664 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:46.820640    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj"
Jul 30 10:54:46.893255 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:46.893224    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a245a488-26a4-4475-8bd0-d7e6d6d1f397-maas-api-service-ca\") pod \"a245a488-26a4-4475-8bd0-d7e6d6d1f397\" (UID: \"a245a488-26a4-4475-8bd0-d7e6d6d1f397\") "
Jul 30 10:54:46.893636 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:46.893614    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a245a488-26a4-4475-8bd0-d7e6d6d1f397-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a245a488-26a4-4475-8bd0-d7e6d6d1f397" (UID: "a245a488-26a4-4475-8bd0-d7e6d6d1f397"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 10:54:46.994529 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:46.994440    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a245a488-26a4-4475-8bd0-d7e6d6d1f397-maas-api-service-ca\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:47.693116 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:47.693080    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj" event={"ID":"a245a488-26a4-4475-8bd0-d7e6d6d1f397","Type":"ContainerDied","Data":"3db0e586679199c4073e54677ed3115bdc4d1addb69f64dc2c99961be5ef0b02"}
Jul 30 10:54:47.693289 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:47.693129    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3db0e586679199c4073e54677ed3115bdc4d1addb69f64dc2c99961be5ef0b02"
Jul 30 10:54:47.693289 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:47.693092    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj"
Jul 30 10:54:48.699691 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.699659    2567 generic.go:356] "Generic (PLEG): container finished" podID="a395fd5f-df04-4b52-94d7-140cf7741d5d" containerID="91a9a712697c505d140d496b5087e05b2322ea8817fb8434052f689c3a376134" exitCode=0
Jul 30 10:54:48.700111 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.699727    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl" event={"ID":"a395fd5f-df04-4b52-94d7-140cf7741d5d","Type":"ContainerDied","Data":"91a9a712697c505d140d496b5087e05b2322ea8817fb8434052f689c3a376134"}
Jul 30 10:54:48.700111 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.699794    2567 scope.go:117] "RemoveContainer" containerID="e6c5b77775952a52b31ac96db0ccbb1e8f13fb6f3fc071940bca9da3f8c48862"
Jul 30 10:54:48.732857 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.732835    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:54:48.809876 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.809788    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-podinfo\") pod \"a395fd5f-df04-4b52-94d7-140cf7741d5d\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") "
Jul 30 10:54:48.809876 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.809827    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-data\") pod \"a395fd5f-df04-4b52-94d7-140cf7741d5d\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") "
Jul 30 10:54:48.809876 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.809853    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a395fd5f-df04-4b52-94d7-140cf7741d5d-istiod-ca-cert\") pod \"a395fd5f-df04-4b52-94d7-140cf7741d5d\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") "
Jul 30 10:54:48.810170 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.809884    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-token\") pod \"a395fd5f-df04-4b52-94d7-140cf7741d5d\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") "
Jul 30 10:54:48.810170 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.810015    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pr755\" (UniqueName: \"kubernetes.io/projected/a395fd5f-df04-4b52-94d7-140cf7741d5d-kube-api-access-pr755\") pod \"a395fd5f-df04-4b52-94d7-140cf7741d5d\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") "
Jul 30 10:54:48.810170 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.810073    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-workload-certs\") pod \"a395fd5f-df04-4b52-94d7-140cf7741d5d\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") "
Jul 30 10:54:48.810170 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.810126    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-envoy\") pod \"a395fd5f-df04-4b52-94d7-140cf7741d5d\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") "
Jul 30 10:54:48.810433 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.810172    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-workload-socket\") pod \"a395fd5f-df04-4b52-94d7-140cf7741d5d\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") "
Jul 30 10:54:48.810433 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.810202    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-credential-socket\") pod \"a395fd5f-df04-4b52-94d7-140cf7741d5d\" (UID: \"a395fd5f-df04-4b52-94d7-140cf7741d5d\") "
Jul 30 10:54:48.810433 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.810382    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a395fd5f-df04-4b52-94d7-140cf7741d5d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a395fd5f-df04-4b52-94d7-140cf7741d5d" (UID: "a395fd5f-df04-4b52-94d7-140cf7741d5d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 10:54:48.810595 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.810555    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a395fd5f-df04-4b52-94d7-140cf7741d5d-istiod-ca-cert\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:48.810652 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.810589    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a395fd5f-df04-4b52-94d7-140cf7741d5d" (UID: "a395fd5f-df04-4b52-94d7-140cf7741d5d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:54:48.810652 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.810633    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a395fd5f-df04-4b52-94d7-140cf7741d5d" (UID: "a395fd5f-df04-4b52-94d7-140cf7741d5d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:54:48.810769 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.810750    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a395fd5f-df04-4b52-94d7-140cf7741d5d" (UID: "a395fd5f-df04-4b52-94d7-140cf7741d5d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:54:48.810839 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.810819    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-data" (OuterVolumeSpecName: "istio-data") pod "a395fd5f-df04-4b52-94d7-140cf7741d5d" (UID: "a395fd5f-df04-4b52-94d7-140cf7741d5d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:54:48.812304 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.812278    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a395fd5f-df04-4b52-94d7-140cf7741d5d" (UID: "a395fd5f-df04-4b52-94d7-140cf7741d5d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:54:48.812412 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.812283    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a395fd5f-df04-4b52-94d7-140cf7741d5d" (UID: "a395fd5f-df04-4b52-94d7-140cf7741d5d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 10:54:48.812623 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.812605    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-token" (OuterVolumeSpecName: "istio-token") pod "a395fd5f-df04-4b52-94d7-140cf7741d5d" (UID: "a395fd5f-df04-4b52-94d7-140cf7741d5d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:54:48.812669 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.812632    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a395fd5f-df04-4b52-94d7-140cf7741d5d-kube-api-access-pr755" (OuterVolumeSpecName: "kube-api-access-pr755") pod "a395fd5f-df04-4b52-94d7-140cf7741d5d" (UID: "a395fd5f-df04-4b52-94d7-140cf7741d5d"). InnerVolumeSpecName "kube-api-access-pr755". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:54:48.911899 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.911846    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-workload-socket\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:48.911899 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.911891    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-credential-socket\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:48.911899 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.911901    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-podinfo\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:48.911899 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.911911    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-data\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:48.911899 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.911922    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-token\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:48.912215 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.911930    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr755\" (UniqueName: \"kubernetes.io/projected/a395fd5f-df04-4b52-94d7-140cf7741d5d-kube-api-access-pr755\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:48.912215 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.911938    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-workload-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:48.912215 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:48.911946    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a395fd5f-df04-4b52-94d7-140cf7741d5d-istio-envoy\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:54:49.705941 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:49.705915    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"
Jul 30 10:54:49.706379 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:49.705966    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl" event={"ID":"a395fd5f-df04-4b52-94d7-140cf7741d5d","Type":"ContainerDied","Data":"0197fe100afef686623182c95b6c2627b8938a194c920d87258435111ddcc4fc"}
Jul 30 10:54:49.706379 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:49.706025    2567 scope.go:117] "RemoveContainer" containerID="91a9a712697c505d140d496b5087e05b2322ea8817fb8434052f689c3a376134"
Jul 30 10:54:49.728789 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:49.728766    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"]
Jul 30 10:54:49.733654 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:49.733632    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-78eb1c-openshift-default-7464f4c85d-mtvtl"]
Jul 30 10:54:50.062658 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:50.062578    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a395fd5f-df04-4b52-94d7-140cf7741d5d" path="/var/lib/kubelet/pods/a395fd5f-df04-4b52-94d7-140cf7741d5d/volumes"
Jul 30 10:54:54.693224 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:54.693190    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj"]
Jul 30 10:54:54.697238 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:54.697216    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2aadf4-xthbj"]
Jul 30 10:54:56.062663 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:56.062625    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a245a488-26a4-4475-8bd0-d7e6d6d1f397" path="/var/lib/kubelet/pods/a245a488-26a4-4475-8bd0-d7e6d6d1f397/volumes"
Jul 30 10:54:57.608773 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:54:57.608734    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:55:01.543253 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:01.543222    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"]
Jul 30 10:55:01.543675 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:01.543481    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8" podUID="8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" containerName="istio-proxy" containerID="cri-o://3573954d736e2e6d70403ead9392a62b940b23e774685fbbc181bc16f1109260" gracePeriod=30
Jul 30 10:55:01.951959 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:01.951921    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt"]
Jul 30 10:55:01.952312 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:01.952299    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a245a488-26a4-4475-8bd0-d7e6d6d1f397" containerName="revoke-keys"
Jul 30 10:55:01.952395 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:01.952314    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a245a488-26a4-4475-8bd0-d7e6d6d1f397" containerName="revoke-keys"
Jul 30 10:55:01.952395 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:01.952325    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a395fd5f-df04-4b52-94d7-140cf7741d5d" containerName="istio-proxy"
Jul 30 10:55:01.952395 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:01.952344    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a395fd5f-df04-4b52-94d7-140cf7741d5d" containerName="istio-proxy"
Jul 30 10:55:01.952492 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:01.952412    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a395fd5f-df04-4b52-94d7-140cf7741d5d" containerName="istio-proxy"
Jul 30 10:55:01.952492 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:01.952420    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a245a488-26a4-4475-8bd0-d7e6d6d1f397" containerName="revoke-keys"
Jul 30 10:55:01.956695 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:01.956674    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt"
Jul 30 10:55:01.959256 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:01.959231    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j6dj6\""
Jul 30 10:55:01.959405 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:01.959239    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 10:55:01.963261 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:01.963232    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt"]
Jul 30 10:55:02.023248 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:02.023215    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/803cef4d-758f-4a84-a5e1-ae8131ca3b11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt\" (UID: \"803cef4d-758f-4a84-a5e1-ae8131ca3b11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt"
Jul 30 10:55:02.124412 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:02.124366    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/803cef4d-758f-4a84-a5e1-ae8131ca3b11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt\" (UID: \"803cef4d-758f-4a84-a5e1-ae8131ca3b11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt"
Jul 30 10:55:02.125016 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:02.124994    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/803cef4d-758f-4a84-a5e1-ae8131ca3b11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt\" (UID: \"803cef4d-758f-4a84-a5e1-ae8131ca3b11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt"
Jul 30 10:55:02.268550 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:02.268459    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt"
Jul 30 10:55:02.399912 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:02.399812    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt"]
Jul 30 10:55:02.402637 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:55:02.402605    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod803cef4d_758f_4a84_a5e1_ae8131ca3b11.slice/crio-32d79f90c6bcd2d11f7ee75a63c71eaef5c1974dcbce589c763186644bef9c17 WatchSource:0}: Error finding container 32d79f90c6bcd2d11f7ee75a63c71eaef5c1974dcbce589c763186644bef9c17: Status 404 returned error can't find the container with id 32d79f90c6bcd2d11f7ee75a63c71eaef5c1974dcbce589c763186644bef9c17
Jul 30 10:55:02.768864 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:02.768828    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt" event={"ID":"803cef4d-758f-4a84-a5e1-ae8131ca3b11","Type":"ContainerStarted","Data":"d837c51aacde394e4e5ef2509aab8dd2afe60f5ad36d0d875ef6a7be8b36c37f"}
Jul 30 10:55:02.769278 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:02.768884    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt" event={"ID":"803cef4d-758f-4a84-a5e1-ae8131ca3b11","Type":"ContainerStarted","Data":"32d79f90c6bcd2d11f7ee75a63c71eaef5c1974dcbce589c763186644bef9c17"}
Jul 30 10:55:02.784737 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:02.784686    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt" podStartSLOduration=1.7846693710000001 podStartE2EDuration="1.784669371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 10:55:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:55:02.783571475 +0000 UTC m=+3939.277766984" watchObservedRunningTime="2026-07-30 10:55:02.784669371 +0000 UTC m=+3939.278864877"
Jul 30 10:55:03.775056 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:03.775020    2567 generic.go:356] "Generic (PLEG): container finished" podID="803cef4d-758f-4a84-a5e1-ae8131ca3b11" containerID="d837c51aacde394e4e5ef2509aab8dd2afe60f5ad36d0d875ef6a7be8b36c37f" exitCode=0
Jul 30 10:55:03.775469 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:03.775094    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt" event={"ID":"803cef4d-758f-4a84-a5e1-ae8131ca3b11","Type":"ContainerDied","Data":"d837c51aacde394e4e5ef2509aab8dd2afe60f5ad36d0d875ef6a7be8b36c37f"}
Jul 30 10:55:04.921319 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:04.921286    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt"
Jul 30 10:55:05.049320 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:05.049237    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/803cef4d-758f-4a84-a5e1-ae8131ca3b11-maas-api-service-ca\") pod \"803cef4d-758f-4a84-a5e1-ae8131ca3b11\" (UID: \"803cef4d-758f-4a84-a5e1-ae8131ca3b11\") "
Jul 30 10:55:05.049653 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:05.049632    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/803cef4d-758f-4a84-a5e1-ae8131ca3b11-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "803cef4d-758f-4a84-a5e1-ae8131ca3b11" (UID: "803cef4d-758f-4a84-a5e1-ae8131ca3b11"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 10:55:05.150622 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:05.150591    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/803cef4d-758f-4a84-a5e1-ae8131ca3b11-maas-api-service-ca\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:55:05.785609 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:05.785571    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt" event={"ID":"803cef4d-758f-4a84-a5e1-ae8131ca3b11","Type":"ContainerDied","Data":"32d79f90c6bcd2d11f7ee75a63c71eaef5c1974dcbce589c763186644bef9c17"}
Jul 30 10:55:05.785609 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:05.785608    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt"
Jul 30 10:55:05.785832 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:05.785630    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="32d79f90c6bcd2d11f7ee75a63c71eaef5c1974dcbce589c763186644bef9c17"
Jul 30 10:55:06.793256 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.793227    2567 generic.go:356] "Generic (PLEG): container finished" podID="8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" containerID="3573954d736e2e6d70403ead9392a62b940b23e774685fbbc181bc16f1109260" exitCode=0
Jul 30 10:55:06.793630 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.793309    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8" event={"ID":"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08","Type":"ContainerDied","Data":"3573954d736e2e6d70403ead9392a62b940b23e774685fbbc181bc16f1109260"}
Jul 30 10:55:06.793630 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.793402    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8" event={"ID":"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08","Type":"ContainerDied","Data":"2bc8eda71daade090e6f682a956177c15dfe144a1fd8bd8da2ec0b0a1e9b6848"}
Jul 30 10:55:06.793630 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.793431    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2bc8eda71daade090e6f682a956177c15dfe144a1fd8bd8da2ec0b0a1e9b6848"
Jul 30 10:55:06.793630 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.793452    2567 scope.go:117] "RemoveContainer" containerID="d82ef21683bec72aaaae30b3a87c78908f01dfdc0951e73bfb8130a1c0cc8607"
Jul 30 10:55:06.802802 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.802785    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:55:06.964292 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.964252    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-podinfo\") pod \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") "
Jul 30 10:55:06.964488 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.964311    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-data\") pod \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") "
Jul 30 10:55:06.964488 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.964362    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-px6f8\" (UniqueName: \"kubernetes.io/projected/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-kube-api-access-px6f8\") pod \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") "
Jul 30 10:55:06.964488 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.964390    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-envoy\") pod \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") "
Jul 30 10:55:06.964488 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.964418    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-workload-certs\") pod \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") "
Jul 30 10:55:06.964488 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.964457    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-workload-socket\") pod \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") "
Jul 30 10:55:06.964751 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.964503    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-credential-socket\") pod \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") "
Jul 30 10:55:06.964751 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.964577    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istiod-ca-cert\") pod \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") "
Jul 30 10:55:06.964751 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.964626    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-token\") pod \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\" (UID: \"8d0089aa-61c9-4d6e-bf75-44ebe7a29b08\") "
Jul 30 10:55:06.964916 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.964852    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" (UID: "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:55:06.964971 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.964902    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" (UID: "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:55:06.965043 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.965021    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-workload-certs\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:55:06.965100 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.965047    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-workload-socket\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:55:06.965218 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.965181    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-data" (OuterVolumeSpecName: "istio-data") pod "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" (UID: "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:55:06.965454 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.965409    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" (UID: "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:55:06.965532 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.965452    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" (UID: "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 10:55:06.966844 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.966820    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" (UID: "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 30 10:55:06.967189 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.967166    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" (UID: "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 10:55:06.967189 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.967180    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-token" (OuterVolumeSpecName: "istio-token") pod "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" (UID: "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:55:06.967355 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:06.967210    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-kube-api-access-px6f8" (OuterVolumeSpecName: "kube-api-access-px6f8") pod "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" (UID: "8d0089aa-61c9-4d6e-bf75-44ebe7a29b08"). InnerVolumeSpecName "kube-api-access-px6f8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 10:55:07.065714 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:07.065670    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-token\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:55:07.065714 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:07.065705    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-podinfo\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:55:07.065714 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:07.065716    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-data\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:55:07.065970 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:07.065728    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-px6f8\" (UniqueName: \"kubernetes.io/projected/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-kube-api-access-px6f8\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:55:07.065970 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:07.065741    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istio-envoy\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:55:07.065970 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:07.065752    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-credential-socket\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:55:07.065970 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:07.065763    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08-istiod-ca-cert\") on node \"ip-10-0-128-146.ec2.internal\" DevicePath \"\""
Jul 30 10:55:07.799763 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:07.799733    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"
Jul 30 10:55:07.837556 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:07.837521    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"]
Jul 30 10:55:07.840699 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:07.840664    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-2aadf4-openshift-default-576c5776f6-bkkm8"]
Jul 30 10:55:08.063664 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:08.063588    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" path="/var/lib/kubelet/pods/8d0089aa-61c9-4d6e-bf75-44ebe7a29b08/volumes"
Jul 30 10:55:12.777589 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:12.777549    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt"]
Jul 30 10:55:12.780140 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:12.780111    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b75946-6x2pt"]
Jul 30 10:55:14.062896 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:14.062861    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="803cef4d-758f-4a84-a5e1-ae8131ca3b11" path="/var/lib/kubelet/pods/803cef4d-758f-4a84-a5e1-ae8131ca3b11/volumes"
Jul 30 10:55:16.108438 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:16.108356    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:55:26.362896 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:26.362854    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bdf164-29756805-57jzd"]
Jul 30 10:55:26.365128 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:26.365095    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-bdf164-29756805-57jzd"]
Jul 30 10:55:28.063189 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:28.063151    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c48ac37-c1d2-4c84-8399-ed075168cdda" path="/var/lib/kubelet/pods/4c48ac37-c1d2-4c84-8399-ed075168cdda/volumes"
Jul 30 10:55:29.608745 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:29.608714    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8xjwm"]
Jul 30 10:55:39.956350 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:39.956308    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-72dg5_d0a38f06-9a44-464b-acbd-6d51e7c2788b/manager/0.log"
Jul 30 10:55:40.070698 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:40.070655    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-wtmnf_ae848eb4-80e1-455f-b4b5-8aa109e02ad9/manager/0.log"
Jul 30 10:55:40.791875 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:40.791848    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d74749788-q6kvh_ba51c3d1-3817-49b8-a583-3bdc4dd70c5c/copy-manifests/0.log"
Jul 30 10:55:40.798277 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:40.798253    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d74749788-q6kvh_ba51c3d1-3817-49b8-a583-3bdc4dd70c5c/manager/0.log"
Jul 30 10:55:41.510634 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:41.510601    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29756805-zwx9k_6b46a103-6400-40c8-987d-a82fa502692f/cleanup/0.log"
Jul 30 10:55:43.694567 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:43.694539    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8xjwm_12ab571e-0521-4dc0-98bd-5ac1f7215401/limitador/0.log"
Jul 30 10:55:43.821260 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:43.821211    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-68zjb_917861bf-8312-496d-93ea-76d8f0679218/manager/0.log"
Jul 30 10:55:45.037047 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:45.037009    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-888465b95-k24rp_b19ed290-2519-455d-8147-688ac9ed11b0/router/0.log"
Jul 30 10:55:45.853867 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:45.853841    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-swzr6_aff02424-d441-4ba2-9304-0745ff5972b0/main/0.log"
Jul 30 10:55:56.246967 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:56.246920    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dfptt_807d7a19-7b79-4963-a013-53d46d57b8a2/global-pull-secret-syncer/0.log"
Jul 30 10:55:56.443870 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:56.443836    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gl5sh_1bbbef53-12ef-475f-934f-5331f026baa6/konnectivity-agent/0.log"
Jul 30 10:55:56.504848 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:55:56.504771    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-146.ec2.internal_963818eb2a7ff168ce7d8b5094579a8c/haproxy/0.log"
Jul 30 10:56:01.932319 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:01.932288    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8xjwm_12ab571e-0521-4dc0-98bd-5ac1f7215401/limitador/0.log"
Jul 30 10:56:02.012418 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:02.012387    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-68zjb_917861bf-8312-496d-93ea-76d8f0679218/manager/0.log"
Jul 30 10:56:04.083985 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:04.083896    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-mz9gf_da4c528d-7534-4727-8ace-474996ab3454/cluster-monitoring-operator/0.log"
Jul 30 10:56:04.434600 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:04.434572    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ss6kg_dc2ff597-4391-4730-a8eb-ba4bfabf04ee/node-exporter/0.log"
Jul 30 10:56:04.455520 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:04.455494    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ss6kg_dc2ff597-4391-4730-a8eb-ba4bfabf04ee/kube-rbac-proxy/0.log"
Jul 30 10:56:04.475801 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:04.475777    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ss6kg_dc2ff597-4391-4730-a8eb-ba4bfabf04ee/init-textfile/0.log"
Jul 30 10:56:06.237788 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.237761    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-fhkxr_8d349f02-e060-4dca-b9b9-44163a0fb935/networking-console-plugin/0.log"
Jul 30 10:56:06.711902 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.711864    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"]
Jul 30 10:56:06.712319 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.712300    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="803cef4d-758f-4a84-a5e1-ae8131ca3b11" containerName="revoke-keys"
Jul 30 10:56:06.712319 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.712319    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="803cef4d-758f-4a84-a5e1-ae8131ca3b11" containerName="revoke-keys"
Jul 30 10:56:06.712473 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.712346    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a395fd5f-df04-4b52-94d7-140cf7741d5d" containerName="istio-proxy"
Jul 30 10:56:06.712473 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.712355    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a395fd5f-df04-4b52-94d7-140cf7741d5d" containerName="istio-proxy"
Jul 30 10:56:06.712473 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.712363    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" containerName="istio-proxy"
Jul 30 10:56:06.712473 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.712370    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" containerName="istio-proxy"
Jul 30 10:56:06.712473 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.712390    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" containerName="istio-proxy"
Jul 30 10:56:06.712473 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.712395    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" containerName="istio-proxy"
Jul 30 10:56:06.712473 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.712453    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="803cef4d-758f-4a84-a5e1-ae8131ca3b11" containerName="revoke-keys"
Jul 30 10:56:06.712473 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.712461    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a395fd5f-df04-4b52-94d7-140cf7741d5d" containerName="istio-proxy"
Jul 30 10:56:06.712473 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.712469    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" containerName="istio-proxy"
Jul 30 10:56:06.712473 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.712476    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d0089aa-61c9-4d6e-bf75-44ebe7a29b08" containerName="istio-proxy"
Jul 30 10:56:06.715654 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.715639    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.719273 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.719250    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-94b2d\"/\"kube-root-ca.crt\""
Jul 30 10:56:06.719389 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.719252    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-94b2d\"/\"openshift-service-ca.crt\""
Jul 30 10:56:06.720286 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.720267    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-94b2d\"/\"default-dockercfg-khmgf\""
Jul 30 10:56:06.727797 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.727776    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"]
Jul 30 10:56:06.884832 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.884784    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bdcf25c9-e21e-416c-8247-9cfafbc88651-podres\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.884832 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.884831    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwg2b\" (UniqueName: \"kubernetes.io/projected/bdcf25c9-e21e-416c-8247-9cfafbc88651-kube-api-access-zwg2b\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.885102 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.884940    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bdcf25c9-e21e-416c-8247-9cfafbc88651-lib-modules\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.885102 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.885021    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bdcf25c9-e21e-416c-8247-9cfafbc88651-sys\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.885102 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.885037    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bdcf25c9-e21e-416c-8247-9cfafbc88651-proc\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.985723 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.985629    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bdcf25c9-e21e-416c-8247-9cfafbc88651-lib-modules\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.985723 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.985695    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bdcf25c9-e21e-416c-8247-9cfafbc88651-sys\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.985723 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.985719    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bdcf25c9-e21e-416c-8247-9cfafbc88651-proc\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.986004 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.985769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bdcf25c9-e21e-416c-8247-9cfafbc88651-podres\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.986004 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.985792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwg2b\" (UniqueName: \"kubernetes.io/projected/bdcf25c9-e21e-416c-8247-9cfafbc88651-kube-api-access-zwg2b\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.986004 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.985818    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bdcf25c9-e21e-416c-8247-9cfafbc88651-proc\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.986004 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.985817    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bdcf25c9-e21e-416c-8247-9cfafbc88651-lib-modules\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.986004 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.985845    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bdcf25c9-e21e-416c-8247-9cfafbc88651-sys\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.986004 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.985888    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bdcf25c9-e21e-416c-8247-9cfafbc88651-podres\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:06.993650 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:06.993627    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwg2b\" (UniqueName: \"kubernetes.io/projected/bdcf25c9-e21e-416c-8247-9cfafbc88651-kube-api-access-zwg2b\") pod \"perf-node-gather-daemonset-wz9rw\" (UID: \"bdcf25c9-e21e-416c-8247-9cfafbc88651\") " pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:07.002250 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:07.002226    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/2.log"
Jul 30 10:56:07.006972 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:07.006952    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-hw45c_97e78a90-a448-4e2d-bdf4-5b21892bb617/console-operator/3.log"
Jul 30 10:56:07.025060 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:07.025043    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:07.162808 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:07.162781    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"]
Jul 30 10:56:07.164836 ip-10-0-128-146 kubenswrapper[2567]: W0730 10:56:07.164806    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbdcf25c9_e21e_416c_8247_9cfafbc88651.slice/crio-fe1960e15c1d28ee2a61a13a2c9194ef11736d49a954da2ee0e3f9f30b8fe790 WatchSource:0}: Error finding container fe1960e15c1d28ee2a61a13a2c9194ef11736d49a954da2ee0e3f9f30b8fe790: Status 404 returned error can't find the container with id fe1960e15c1d28ee2a61a13a2c9194ef11736d49a954da2ee0e3f9f30b8fe790
Jul 30 10:56:08.101732 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:08.101692    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw" event={"ID":"bdcf25c9-e21e-416c-8247-9cfafbc88651","Type":"ContainerStarted","Data":"a2e3d9d04374e4450dcc11a3d6cd7f5a0095f76d3d3827c9b9427259880c3869"}
Jul 30 10:56:08.102162 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:08.101747    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw" event={"ID":"bdcf25c9-e21e-416c-8247-9cfafbc88651","Type":"ContainerStarted","Data":"fe1960e15c1d28ee2a61a13a2c9194ef11736d49a954da2ee0e3f9f30b8fe790"}
Jul 30 10:56:08.102162 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:08.101790    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:08.120109 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:08.120059    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw" podStartSLOduration=2.120044326 podStartE2EDuration="2.120044326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 10:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 10:56:08.11836776 +0000 UTC m=+4004.612563268" watchObservedRunningTime="2026-07-30 10:56:08.120044326 +0000 UTC m=+4004.614239832"
Jul 30 10:56:08.221008 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:08.220976    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-tbk9v_e5551d48-3f70-4f9a-80eb-9daf2a25cf6d/volume-data-source-validator/0.log"
Jul 30 10:56:09.305287 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:09.305262    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-84b5h_3c1a1aad-170c-4e52-8f7d-e763663b81e4/dns/0.log"
Jul 30 10:56:09.325566 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:09.325540    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-84b5h_3c1a1aad-170c-4e52-8f7d-e763663b81e4/kube-rbac-proxy/0.log"
Jul 30 10:56:09.420085 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:09.420052    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gqhzm_2cfa238d-618e-4dc2-9128-77df904614c9/dns-node-resolver/0.log"
Jul 30 10:56:10.044060 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:10.044017    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lt2xc_f556c8a9-3801-4f40-b5d7-1ffa2701eed1/node-ca/0.log"
Jul 30 10:56:11.418855 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:11.418830    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-888465b95-k24rp_b19ed290-2519-455d-8147-688ac9ed11b0/router/0.log"
Jul 30 10:56:12.270252 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:12.270210    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zhg8c_c89418e7-06ea-4bb2-8c58-22fbd57034b5/serve-healthcheck-canary/0.log"
Jul 30 10:56:12.751093 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:12.751060    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-p8h62_353dc841-0a51-4587-b4a9-6a4955ad1a12/insights-operator/0.log"
Jul 30 10:56:12.752973 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:12.752955    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-p8h62_353dc841-0a51-4587-b4a9-6a4955ad1a12/insights-operator/1.log"
Jul 30 10:56:12.839107 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:12.839080    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cr8nk_ca650cbb-da82-43ee-b3fe-397ab13a2a25/kube-rbac-proxy/0.log"
Jul 30 10:56:12.859654 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:12.859629    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cr8nk_ca650cbb-da82-43ee-b3fe-397ab13a2a25/exporter/0.log"
Jul 30 10:56:12.887416 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:12.887386    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cr8nk_ca650cbb-da82-43ee-b3fe-397ab13a2a25/extractor/0.log"
Jul 30 10:56:14.116880 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:14.116852    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-94b2d/perf-node-gather-daemonset-wz9rw"
Jul 30 10:56:15.213105 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:15.213073    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-72dg5_d0a38f06-9a44-464b-acbd-6d51e7c2788b/manager/0.log"
Jul 30 10:56:15.437118 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:15.437089    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-wtmnf_ae848eb4-80e1-455f-b4b5-8aa109e02ad9/manager/0.log"
Jul 30 10:56:15.968568 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:15.968535    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d74749788-q6kvh_ba51c3d1-3817-49b8-a583-3bdc4dd70c5c/manager/0.log"
Jul 30 10:56:16.001005 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:16.000975    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d74749788-q6kvh_ba51c3d1-3817-49b8-a583-3bdc4dd70c5c/copy-manifests/0.log"
Jul 30 10:56:17.342676 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:17.342646    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-75b6964665-2rpdh_fabd841c-cd79-4c2c-b74a-ca48acf65cdc/manager/0.log"
Jul 30 10:56:17.815265 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:17.815234    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-888465b95-k24rp_b19ed290-2519-455d-8147-688ac9ed11b0/router/0.log"
Jul 30 10:56:19.458659 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:19.458629    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-888465b95-k24rp_b19ed290-2519-455d-8147-688ac9ed11b0/router/0.log"
Jul 30 10:56:22.535868 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:22.535838    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-pmclw_f06a6ff0-0ed2-4918-b283-081db15b9418/migrator/0.log"
Jul 30 10:56:22.554663 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:22.554638    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-pmclw_f06a6ff0-0ed2-4918-b283-081db15b9418/graceful-termination/0.log"
Jul 30 10:56:22.911748 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:22.911716    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r_a79aad4c-afde-4ab7-b54a-4fd145a20252/kube-storage-version-migrator-operator/1.log"
Jul 30 10:56:22.912673 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:22.912644    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-xlh8r_a79aad4c-afde-4ab7-b54a-4fd145a20252/kube-storage-version-migrator-operator/0.log"
Jul 30 10:56:24.150666 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:24.150634    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-98kcx_e69de26a-c120-4b24-a283-04e2db290ed4/kube-multus-additional-cni-plugins/0.log"
Jul 30 10:56:24.172386 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:24.172359    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-98kcx_e69de26a-c120-4b24-a283-04e2db290ed4/egress-router-binary-copy/0.log"
Jul 30 10:56:24.192115 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:24.192091    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-98kcx_e69de26a-c120-4b24-a283-04e2db290ed4/cni-plugins/0.log"
Jul 30 10:56:24.211695 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:24.211673    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-98kcx_e69de26a-c120-4b24-a283-04e2db290ed4/bond-cni-plugin/0.log"
Jul 30 10:56:24.233345 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:24.233306    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-98kcx_e69de26a-c120-4b24-a283-04e2db290ed4/routeoverride-cni/0.log"
Jul 30 10:56:24.252551 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:24.252525    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-98kcx_e69de26a-c120-4b24-a283-04e2db290ed4/whereabouts-cni-bincopy/0.log"
Jul 30 10:56:24.273495 ip-10-0-128-146 kubenswrapper[2567]: I0730 10:56:24.273464    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-98kcx_e69de26a-c120-4b24-a283-04e2db290ed4/whereabouts-cni/0.log"