Jul 28 01:02:01.291010 ip-10-0-138-173 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 01:02:01.291021 ip-10-0-138-173 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 01:02:01.291029 ip-10-0-138-173 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 01:02:01.291292 ip-10-0-138-173 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 01:02:11.310229 ip-10-0-138-173 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 01:02:11.310242 ip-10-0-138-173 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 0794526eddbc45d79a5d0b5365d83b34 --
Jul 28 01:04:07.748243 ip-10-0-138-173 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 01:04:08.135091 ip-10-0-138-173 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 01:04:08.135091 ip-10-0-138-173 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 01:04:08.135091 ip-10-0-138-173 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 01:04:08.135091 ip-10-0-138-173 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 01:04:08.135091 ip-10-0-138-173 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 01:04:08.136680 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.136565    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 01:04:08.145647 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145626    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 01:04:08.145647 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145643    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 01:04:08.145647 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145647    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 01:04:08.145647 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145650    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 01:04:08.145647 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145653    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 01:04:08.145647 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145655    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145658    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145661    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145664    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145666    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145669    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145672    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145674    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145677    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145679    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145682    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145684    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145687    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145689    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145692    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145695    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145697    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145699    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145704    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 01:04:08.145865 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145708    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145711    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145713    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145716    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145719    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145721    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145724    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145727    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145729    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145732    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145734    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145737    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145739    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145742    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145744    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145747    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145750    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145752    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145755    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145759    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 01:04:08.146336 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145761    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145763    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145766    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145769    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145771    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145774    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145777    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145780    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145783    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145785    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145788    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145790    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145793    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145795    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145797    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145800    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145802    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145806    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145810    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145814    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 01:04:08.146863 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145816    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145819    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145821    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145824    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145826    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145830    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145833    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145836    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145839    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145843    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145846    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145850    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145853    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145856    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145859    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145861    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145864    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145866    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145869    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 01:04:08.147381 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145872    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145875    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145878    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145880    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145883    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145885    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145888    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145891    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145893    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145895    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145898    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145900    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145903    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145906    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145908    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.145911    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146603    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146610    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146613    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146615    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 01:04:08.147822 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146618    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146621    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146623    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146626    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146629    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146631    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146634    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146639    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146643    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146646    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146649    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146652    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146654    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146657    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146661    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146663    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146666    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146668    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146671    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 01:04:08.148311 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146673    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146676    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146679    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146682    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146684    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146687    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146690    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146693    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146695    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146698    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146701    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146703    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146706    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146709    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146711    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146714    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146717    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146719    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146722    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 01:04:08.148775 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146725    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146729    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146733    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146736    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146739    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146742    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146745    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146748    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146751    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146753    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146756    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146758    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146761    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146764    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146766    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146769    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146772    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146775    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146778    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 01:04:08.149285 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146781    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146784    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146787    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146789    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146792    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146795    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146800    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146802    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146805    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146807    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146810    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146812    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146815    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146817    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146820    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146823    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146825    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146827    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146830    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146832    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 01:04:08.149735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146835    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146838    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146841    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146843    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146846    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146848    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146851    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146853    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146856    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146858    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146861    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146864    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146866    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146869    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146872    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146875    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146877    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.146880    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147276    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147281    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 01:04:08.150236 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147284    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147287    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147290    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147292    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147295    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147297    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147300    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147302    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147305    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147309    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147312    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147315    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147318    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147320    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147323    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147326    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147329    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147331    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147333    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147336    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 01:04:08.150712 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147340    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147343    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147346    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147348    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147351    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147354    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147357    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147361    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147365    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147368    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147372    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147374    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147377    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147380    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147382    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147385    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147388    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147391    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147394    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147396    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 01:04:08.151230 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147398    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147401    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147404    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147407    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147409    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147411    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147414    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147417    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147421    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147424    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147427    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147430    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147434    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147437    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147439    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147442    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147444    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147447    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147449    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 01:04:08.151701 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147452    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147454    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147457    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147461    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147463    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147466    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147469    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147471    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147473    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147476    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147478    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147481    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147483    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147486    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147488    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147491    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147493    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147496    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147498    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 01:04:08.152158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147501    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147503    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147506    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147509    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147511    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147514    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147518    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147520    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147523    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147525    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147528    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147530    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147533    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147535    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147537    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147540    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147544    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147547    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.147549    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147617    2565 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147623    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 01:04:08.152605 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147630    2565 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147635    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147638    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147641    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147645    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147649    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147652    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147655    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147658    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147661    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147664    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147667    2565 flags.go:64] FLAG: --cgroup-root=""
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147670    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147673    2565 flags.go:64] FLAG: --client-ca-file=""
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147676    2565 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147679    2565 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147683    2565 flags.go:64] FLAG: --cluster-domain=""
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147686    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147690    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147692    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147696    2565 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147699    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147702    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 01:04:08.153123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147705    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147708    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147711    2565 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147713    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147718    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147721    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147723    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147727    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147730    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147733    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147736    2565 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147739    2565 flags.go:64] FLAG: --enable-server="true"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147742    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147746    2565 flags.go:64] FLAG: --event-burst="100"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147749    2565 flags.go:64] FLAG: --event-qps="50"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147752    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147755    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147758    2565 flags.go:64] FLAG: --eviction-hard=""
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147761    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147764    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147767    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147770    2565 flags.go:64] FLAG: --eviction-soft=""
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147773    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147776    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147779    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 01:04:08.153666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147782    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147786    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147789    2565 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147792    2565 flags.go:64] FLAG: --feature-gates=""
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147798    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147800    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147803    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147806    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147809    2565 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147812    2565 flags.go:64] FLAG: --help="false"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147814    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147818    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147821    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147824    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147828    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147832    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147835    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147838    2565 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147840    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147843    2565 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147846    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147849    2565 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147851    2565 flags.go:64] FLAG: --kube-reserved=""
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147855    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 01:04:08.154323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147857    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147860    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147863    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147865    2565 flags.go:64] FLAG: --lock-file=""
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147868    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147871    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147874    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147879    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147882    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147886    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147889    2565 flags.go:64] FLAG: --logging-format="text"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147891    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147894    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147897    2565 flags.go:64] FLAG: --manifest-url=""
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147900    2565 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147904    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147907    2565 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147911    2565 flags.go:64] FLAG: --max-pods="110"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147914    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147917    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147920    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147923    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147926    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147929    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147932    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 01:04:08.154908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147939    2565 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147942    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147945    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147948    2565 flags.go:64] FLAG: --pod-cidr=""
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147951    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147974    2565 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147978    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147981    2565 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147984    2565 flags.go:64] FLAG: --port="10250"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147987    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147989    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-024b0ffedb27650b1"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147992    2565 flags.go:64] FLAG: --qos-reserved=""
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147996    2565 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.147998    2565 flags.go:64] FLAG: --register-node="true"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148001    2565 flags.go:64] FLAG: --register-with-taints=""
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148009    2565 flags.go:64] FLAG: --registry-burst="10"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148013    2565 flags.go:64] FLAG: --registry-qps="5"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148017    2565 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148020    2565 flags.go:64] FLAG: --reserved-memory=""
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148023    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148026    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148029    2565 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148032    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148035    2565 flags.go:64] FLAG: --runonce="false"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148038    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 01:04:08.155601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148041    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148044    2565 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148047    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148050    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148053    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148056    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148058    2565 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148061    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148064    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148067    2565 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148069    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148072    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148075    2565 flags.go:64] FLAG: --system-cgroups=""
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148078    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148083    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148086    2565 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148088    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148092    2565 flags.go:64] FLAG: --tls-min-version=""
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148095    2565 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148098    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148100    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148103    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148106    2565 flags.go:64] FLAG: --v="2"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148109    2565 flags.go:64] FLAG: --version="false"
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148116    2565 flags.go:64] FLAG: --vmodule=""
Jul 28 01:04:08.156285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148120    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148123    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148260    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148264    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148267    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148270    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148272    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148275    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148280    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148282    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148285    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148288    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148291    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148293    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148295    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148298    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148300    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148303    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148305    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148307    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 01:04:08.156899 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148310    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148313    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148315    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148319    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148323    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148325    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148328    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148331    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148334    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148336    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148339    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148343    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148346    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148348    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148351    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148354    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148356    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148359    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148361    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148364    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 01:04:08.157397 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148367    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148370    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148374    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148378    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148380    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148383    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148385    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148388    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148390    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148393    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148395    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148398    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148401    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148403    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148405    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148408    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148410    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148413    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148415    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148418    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 01:04:08.157862 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148420    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148422    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148425    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148431    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148433    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148436    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148438    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148441    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148443    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148445    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148448    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148450    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148454    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148457    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148460    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148463    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148467    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148469    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148472    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148475    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 01:04:08.158387 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148477    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148480    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148482    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148485    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148487    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148490    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148492    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148494    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148497    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148499    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148502    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148504    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148507    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148509    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148511    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148515    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148517    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148520    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148523    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 01:04:08.158857 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148525    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 01:04:08.159331 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.148528    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 01:04:08.159331 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.148533    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 01:04:08.159331 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.155434    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 01:04:08.159331 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.155448    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 01:04:08.159331 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.155467    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 01:04:08.159331 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.155473    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 01:04:08.159331 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155528    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 01:04:08.159331 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155533    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 01:04:08.159331 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155536    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 01:04:08.159331 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155539    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 01:04:08.159331 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155541    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 01:04:08.159331 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155544    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 01:04:08.159331 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155547    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 01:04:08.159331 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155549    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155552    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155554    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155557    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155560    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155562    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155565    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155567    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155570    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155572    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155575    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155577    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155580    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155583    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155586    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155589    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155591    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155594    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155597    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155599    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 01:04:08.159665 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155602    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155604    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155608    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155611    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155614    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155617    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155621    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155624    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155627    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155630    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155633    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155637    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155640    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155642    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155645    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155647    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155649    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155652    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155655    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155657    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 01:04:08.160185 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155660    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155662    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155665    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155667    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155670    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155672    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155675    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155678    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155681    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155683    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155686    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155688    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155691    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155694    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155696    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155699    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155702    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155704    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155707    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155709    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 01:04:08.160666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155713    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155716    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155718    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155721    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155723    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155726    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155728    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155730    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155733    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155735    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155737    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155740    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155743    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155745    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155748    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155750    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155753    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155755    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155758    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155761    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 01:04:08.161156 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155763    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 01:04:08.161652 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155765    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 01:04:08.161652 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155768    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 01:04:08.161652 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155770    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 01:04:08.161652 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155773    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 01:04:08.161652 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155776    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 01:04:08.161652 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155779    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 01:04:08.161652 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155782    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 01:04:08.161652 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155784    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 01:04:08.161652 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155787    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 01:04:08.161652 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155791    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 01:04:08.161652 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155794    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 01:04:08.161652 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.155799    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 01:04:08.161652 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155914    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 01:04:08.161652 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155919    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155922    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155925    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155927    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155929    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155932    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155935    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155937    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155940    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155942    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155945    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155948    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155950    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155953    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155969    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155972    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155975    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155978    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155980    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 01:04:08.162004 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155983    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155986    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155988    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155991    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155993    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155996    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.155999    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156001    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156004    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156006    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156009    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156012    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156015    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156018    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156021    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156023    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156026    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156028    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156031    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156033    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 01:04:08.162454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156035    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156038    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156040    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156043    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156045    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156047    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156050    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156052    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156055    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156057    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156060    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156063    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156065    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156068    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156070    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156072    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156075    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156077    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156080    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156083    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 01:04:08.162948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156085    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156088    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156090    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156092    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156095    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156097    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156100    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156102    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156105    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156108    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156112    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156114    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156117    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156120    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156123    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156126    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156130    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156133    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156135    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 01:04:08.163438 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156138    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156141    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156144    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156146    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156149    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156151    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156154    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156157    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156159    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156162    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156164    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156167    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156169    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156171    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156174    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156177    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156179    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156182    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156184    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 01:04:08.163880 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:08.156187    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 01:04:08.164350 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.156192    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 01:04:08.164350 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.156282    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 01:04:08.164350 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.158688    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 01:04:08.164350 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.160153    2565 server.go:1019] "Starting client certificate rotation"
Jul 28 01:04:08.164350 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.160253    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 01:04:08.164350 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.160285    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 01:04:08.164350 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.162763    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 01:04:08.165922 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.165909    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 01:04:08.174779 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.174761    2565 log.go:25] "Validated CRI v1 runtime API"
Jul 28 01:04:08.178224 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.178208    2565 log.go:25] "Validated CRI v1 image API"
Jul 28 01:04:08.179184 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.179163    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 01:04:08.181463 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.181446    2565 fs.go:135] Filesystem UUIDs: map[249e07e9-3b68-4ffa-8ceb-8c65a997d1f6:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 cfa286b4-c797-4b24-8ec5-7f8711ce2526:/dev/nvme0n1p3]
Jul 28 01:04:08.181528 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.181464    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 01:04:08.186251 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.186146    2565 manager.go:216] Machine: {Timestamp:2026-07-28 01:04:08.185066993 +0000 UTC m=+0.321485010 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3094853 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e428cd35df00bf8aa6d1eb9f7f5cf SystemUUID:ec2e428c-d35d-f00b-f8aa-6d1eb9f7f5cf BootID:0794526e-ddbc-45d7-9a5d-0b5365d83b34 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e8:e6:f8:b3:c5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e8:e6:f8:b3:c5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:fa:b1:65:69:90 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 01:04:08.186251 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.186249    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 01:04:08.186356 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.186320    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 01:04:08.187548 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.187523    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 01:04:08.187672 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.187551    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-173.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 01:04:08.188990 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.188980    2565 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 01:04:08.189030 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.188993    2565 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 01:04:08.189030 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.189000    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 01:04:08.189804 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.189795    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 01:04:08.190861 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.190851    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 01:04:08.191698 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.191683    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 01:04:08.192692 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.192680    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 01:04:08.192835 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.192825    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 01:04:08.192865 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.192855    2565 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 01:04:08.192865 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.192864    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 01:04:08.192922 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.192875    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 01:04:08.192922 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.192884    2565 kubelet.go:398] "Adding apiserver pod source"
Jul 28 01:04:08.192922 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.192893    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 01:04:08.193224 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.193212    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 01:04:08.193288 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.193231    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 01:04:08.195178 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.195157    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 01:04:08.197493 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.197479    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 01:04:08.197553 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.197508    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 01:04:08.199553 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.199542    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 01:04:08.199593 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.199559    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 01:04:08.199593 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.199566    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 01:04:08.199593 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.199571    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 01:04:08.199593 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.199582    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 01:04:08.199593 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.199591    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 01:04:08.199721 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.199602    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 01:04:08.199721 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.199608    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 01:04:08.199721 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.199615    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 01:04:08.199721 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.199620    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 01:04:08.199721 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.199628    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 01:04:08.199721 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.199633    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 01:04:08.200358 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.200348    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 01:04:08.200387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.200361    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 01:04:08.202047 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.202028    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-173.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 01:04:08.202141 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.202119    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 01:04:08.202279 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.202259    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 01:04:08.203510 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.203498    2565 server.go:1268] "Started kubelet"
Jul 28 01:04:08.203612 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.203580    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 01:04:08.203712 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.203625    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 01:04:08.203712 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.203591    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 01:04:08.204404 ip-10-0-138-173 systemd[1]: Started Kubernetes Kubelet.
Jul 28 01:04:08.206593 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.206571    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 01:04:08.208366 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.208340    2565 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 01:04:08.209312 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.209283    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 01:04:08.211413 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.210546    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-173.ec2.internal.18c64e87f8f6fee2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-173.ec2.internal,UID:ip-10-0-138-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-173.ec2.internal,},FirstTimestamp:2026-07-28 01:04:08.203468514 +0000 UTC m=+0.339886532,LastTimestamp:2026-07-28 01:04:08.203468514 +0000 UTC m=+0.339886532,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-173.ec2.internal,}"
Jul 28 01:04:08.222613 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.222600    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 01:04:08.222686 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.222619    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 01:04:08.222728 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.222689    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-173.ec2.internal\" not found"
Jul 28 01:04:08.223543 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.223441    2565 factory.go:55] Registering systemd factory
Jul 28 01:04:08.223543 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.223456    2565 factory.go:223] Registration of the systemd container factory successfully
Jul 28 01:04:08.223699 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.223675    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 01:04:08.223796 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.223780    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 01:04:08.223796 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.223794    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 01:04:08.223911 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.223832    2565 factory.go:153] Registering CRI-O factory
Jul 28 01:04:08.223911 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.223847    2565 factory.go:223] Registration of the crio container factory successfully
Jul 28 01:04:08.223911 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.223894    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 01:04:08.224046 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.223912    2565 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 01:04:08.224046 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.223920    2565 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 01:04:08.224046 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.223920    2565 factory.go:103] Registering Raw factory
Jul 28 01:04:08.224046 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.223953    2565 manager.go:1196] Started watching for new ooms in manager
Jul 28 01:04:08.224582 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.224569    2565 manager.go:349] Starting recovery of all containers
Jul 28 01:04:08.232671 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.232654    2565 manager.go:354] Recovery completed
Jul 28 01:04:08.233378 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.233352    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 01:04:08.233471 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.233357    2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 01:04:08.236657 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.236554    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:08.237052 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.237026    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w7xn4"
Jul 28 01:04:08.238452 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.238435    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:08.238527 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.238468    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:08.238527 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.238482    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:08.239010 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.238991    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 01:04:08.239010 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.239008    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 01:04:08.239144 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.239028    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 01:04:08.239144 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.239001    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:08.239722 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.239708    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:08.239766 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.239733    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:08.239766 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.239743    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:08.241047 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.241035    2565 policy_none.go:49] "None policy: Start"
Jul 28 01:04:08.241092 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.241054    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 01:04:08.241092 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.241064    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 01:04:08.242261 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.242242    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w7xn4"
Jul 28 01:04:08.243945 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.243934    2565 policy_none.go:47] "Start"
Jul 28 01:04:08.248265 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.248184    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-173.ec2.internal.18c64e87fb0ccac9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-173.ec2.internal,UID:ip-10-0-138-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-173.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-173.ec2.internal,},FirstTimestamp:2026-07-28 01:04:08.238451401 +0000 UTC m=+0.374869422,LastTimestamp:2026-07-28 01:04:08.238451401 +0000 UTC m=+0.374869422,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-173.ec2.internal,}"
Jul 28 01:04:08.288497 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.288480    2565 manager.go:337] "Starting Device Plugin manager"
Jul 28 01:04:08.288592 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.288514    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 01:04:08.288592 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.288526    2565 server.go:87] "Starting device plugin registration server"
Jul 28 01:04:08.288748 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.288735    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 01:04:08.288795 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.288749    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 01:04:08.288886 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.288870    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 01:04:08.288990 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.288950    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 01:04:08.288990 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.288980    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 01:04:08.289422 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.289404    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 01:04:08.289489 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.289455    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-173.ec2.internal\" not found"
Jul 28 01:04:08.367788 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.367753    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 01:04:08.369023 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.369008    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 01:04:08.369107 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.369049    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 01:04:08.369107 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.369077    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 01:04:08.369208 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.369115    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 01:04:08.371638 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.371620    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:04:08.389300 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.389257    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:08.390064 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.390040    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:08.390064 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.390066    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:08.390180 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.390075    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:08.390180 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.390098    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.398015 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.397997    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.398060 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.398018    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-173.ec2.internal\": node \"ip-10-0-138-173.ec2.internal\" not found"
Jul 28 01:04:08.418191 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.418174    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-173.ec2.internal\" not found"
Jul 28 01:04:08.470205 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.470186    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-173.ec2.internal"]
Jul 28 01:04:08.470279 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.470251    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:08.471789 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.471772    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:08.471872 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.471800    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:08.471872 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.471809    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:08.472985 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.472971    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:08.473113 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.473087    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.473170 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.473133    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:08.473835 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.473818    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:08.473913 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.473848    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:08.473913 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.473827    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:08.473913 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.473859    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:08.473913 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.473872    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:08.473913 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.473887    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:08.475135 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.475119    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.475215 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.475149    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:08.475757 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.475742    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:08.475830 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.475768    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:08.475830 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.475780    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:08.492491 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.492473    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-173.ec2.internal\" not found" node="ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.496317 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.496300    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-173.ec2.internal\" not found" node="ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.518479 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.518459    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-173.ec2.internal\" not found"
Jul 28 01:04:08.525128 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.525108    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f5920e338c0b1aa0d84686639273f07-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal\" (UID: \"1f5920e338c0b1aa0d84686639273f07\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.525199 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.525130    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f5920e338c0b1aa0d84686639273f07-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal\" (UID: \"1f5920e338c0b1aa0d84686639273f07\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.525199 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.525147    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/415ae5ae7d5d3fc074b4ce415469a19e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-173.ec2.internal\" (UID: \"415ae5ae7d5d3fc074b4ce415469a19e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.619382 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.619363    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-173.ec2.internal\" not found"
Jul 28 01:04:08.625726 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.625706    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f5920e338c0b1aa0d84686639273f07-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal\" (UID: \"1f5920e338c0b1aa0d84686639273f07\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.625796 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.625738    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f5920e338c0b1aa0d84686639273f07-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal\" (UID: \"1f5920e338c0b1aa0d84686639273f07\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.625796 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.625762    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/415ae5ae7d5d3fc074b4ce415469a19e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-173.ec2.internal\" (UID: \"415ae5ae7d5d3fc074b4ce415469a19e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.625888 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.625810    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1f5920e338c0b1aa0d84686639273f07-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal\" (UID: \"1f5920e338c0b1aa0d84686639273f07\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.625888 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.625814    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/415ae5ae7d5d3fc074b4ce415469a19e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-173.ec2.internal\" (UID: \"415ae5ae7d5d3fc074b4ce415469a19e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.625888 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.625816    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f5920e338c0b1aa0d84686639273f07-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal\" (UID: \"1f5920e338c0b1aa0d84686639273f07\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.720148 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.720089    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-173.ec2.internal\" not found"
Jul 28 01:04:08.795594 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.795565    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.801795 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:08.801779    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:08.820476 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.820451    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-173.ec2.internal\" not found"
Jul 28 01:04:08.921320 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:08.921282    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-173.ec2.internal\" not found"
Jul 28 01:04:10.113457 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:10.113378    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-173.ec2.internal\" not found"
Jul 28 01:04:10.213934 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:10.213899    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-173.ec2.internal\" not found"
Jul 28 01:04:10.251403 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.251374    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 01:04:10.251867 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.251607    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 01:04:10.314223 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:10.314195    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-173.ec2.internal\" not found"
Jul 28 01:04:10.339079 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.339049    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 00:59:09 +0000 UTC" deadline="2028-04-11 12:19:16.327487482 +0000 UTC"
Jul 28 01:04:10.339079 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.339074    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14963h15m5.988417976s"
Jul 28 01:04:10.386393 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.386334    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:10.387318 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.387298    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:10.387410 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.387333    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:10.387410 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.387346    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:10.414401 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:10.414382    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-173.ec2.internal\" not found"
Jul 28 01:04:10.514507 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:10.514468    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-173.ec2.internal\" not found"
Jul 28 01:04:10.593200 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.593184    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:04:10.615159 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.615140    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:10.627044 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.627029    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 01:04:10.628757 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.628744    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-173.ec2.internal"
Jul 28 01:04:10.632245 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.632231    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:04:10.636688 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.636645    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 01:04:10.702778 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:10.702759    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:04:11.039186 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:11.039141    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f5920e338c0b1aa0d84686639273f07.slice/crio-f9586f69682bf2c6a50351966d2e57c3b49c2ae44e6e837567226d704846fc05 WatchSource:0}: Error finding container f9586f69682bf2c6a50351966d2e57c3b49c2ae44e6e837567226d704846fc05: Status 404 returned error can't find the container with id f9586f69682bf2c6a50351966d2e57c3b49c2ae44e6e837567226d704846fc05
Jul 28 01:04:11.039463 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:11.039448    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod415ae5ae7d5d3fc074b4ce415469a19e.slice/crio-ce7082f01d4ec7372cad19afad3f18a0ddc1853e22b4ca453f1b11de90d40331 WatchSource:0}: Error finding container ce7082f01d4ec7372cad19afad3f18a0ddc1853e22b4ca453f1b11de90d40331: Status 404 returned error can't find the container with id ce7082f01d4ec7372cad19afad3f18a0ddc1853e22b4ca453f1b11de90d40331
Jul 28 01:04:11.043411 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.043397    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:04:11.285822 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.285750    2565 apiserver.go:52] "Watching apiserver"
Jul 28 01:04:11.295615 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.295588    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 01:04:11.295926 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.295905    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-rk9c4","openshift-network-operator/iptables-alerter-xlb4s","openshift-ovn-kubernetes/ovnkube-node-5qk5p","kube-system/konnectivity-agent-6p8r2","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv","openshift-cluster-node-tuning-operator/tuned-4pjgt","openshift-dns/node-resolver-w4hf9","openshift-image-registry/node-ca-dg6qv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal","openshift-multus/multus-74qj2","openshift-multus/multus-additional-cni-plugins-xx4zt","kube-system/global-pull-secret-syncer-8k57t","kube-system/kube-apiserver-proxy-ip-10-0-138-173.ec2.internal","openshift-multus/network-metrics-daemon-g8zj6"]
Jul 28 01:04:11.297802 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.297782    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w4hf9"
Jul 28 01:04:11.299130 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.299102    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.300136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.300120    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-v9ckx\""
Jul 28 01:04:11.300340 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.300326    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.300598 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.300520    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 01:04:11.300598 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.300580    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 01:04:11.301570 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.301550    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6p8r2"
Jul 28 01:04:11.301981 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.301937    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 01:04:11.302061 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.301995    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 01:04:11.302061 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.302026    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 01:04:11.302061 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.301970    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 01:04:11.302204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.302122    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8fmqh\""
Jul 28 01:04:11.302204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.302192    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 01:04:11.302805 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.302793    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.305714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.303697    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 01:04:11.305714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.303726    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 01:04:11.305714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.303728    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 01:04:11.305714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.303782    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 01:04:11.305714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.304071    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.305714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.305187    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 01:04:11.305714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.305250    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 01:04:11.305714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.305316    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 01:04:11.305714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.305333    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x4597\""
Jul 28 01:04:11.305714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.305382    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-f9d7t\""
Jul 28 01:04:11.305714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.305512    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 01:04:11.305714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.305560    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 01:04:11.305714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.305656    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 01:04:11.305714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.305713    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:11.306347 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.305782    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:11.306347 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.305793    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zhhkc\""
Jul 28 01:04:11.306347 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.305797    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 01:04:11.306668 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.306651    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dg6qv"
Jul 28 01:04:11.306778 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.306763    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9qwtx\""
Jul 28 01:04:11.306839 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.306809    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 01:04:11.306930 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.306916    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:04:11.307898 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.307884    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xlb4s"
Jul 28 01:04:11.308848 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.308831    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lnhhg\""
Jul 28 01:04:11.309107 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.309086    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.309199 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.309142    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 01:04:11.309199 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.309157    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 01:04:11.309299 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.309226    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 01:04:11.310270 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.310253    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:11.310364 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.310314    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:11.310655 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.310635    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8grww\""
Jul 28 01:04:11.310655 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.310643    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:04:11.310795 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.310659    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 01:04:11.310795 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.310665    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 01:04:11.313842 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.311933    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vwvpj\""
Jul 28 01:04:11.313842 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.311973    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:11.313842 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.312049    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:11.314036 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.314017    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 01:04:11.314463 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.314442    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 01:04:11.316048 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.316032    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 01:04:11.324859 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.324840    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:04:11.325581 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325567    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-etc-selinux\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.325649 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325589    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:11.325649 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325606    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfs4m\" (UniqueName: \"kubernetes.io/projected/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-kube-api-access-wfs4m\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.325649 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325620    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-log-socket\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.325649 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325635    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5b3d5029-ddde-4549-938d-63c6102bea7b-iptables-alerter-script\") pod \"iptables-alerter-xlb4s\" (UID: \"5b3d5029-ddde-4549-938d-63c6102bea7b\") " pod="openshift-network-operator/iptables-alerter-xlb4s"
Jul 28 01:04:11.325782 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325652    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-sysconfig\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.325782 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325692    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-host\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.325782 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325716    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-cnibin\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.325782 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325731    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-etc-kubernetes\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.325782 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325747    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.325782 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325765    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-cni-bin\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.325782 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325779    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-sys-fs\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.326108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325791    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d4fed09-210b-4210-8be1-0edd73941e11-tmp\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.326108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325824    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-hostroot\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.326108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325854    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-cni-netd\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.326108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325879    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g7f7\" (UniqueName: \"kubernetes.io/projected/f02709be-e454-4322-b57b-333a63c4093f-kube-api-access-8g7f7\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.326108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325904    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f874e33-77cd-4288-abd3-77433942b384-host\") pod \"node-ca-dg6qv\" (UID: \"6f874e33-77cd-4288-abd3-77433942b384\") " pod="openshift-image-registry/node-ca-dg6qv"
Jul 28 01:04:11.326108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325929    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-run-k8s-cni-cncf-io\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.326108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325952    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-run-netns\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.326108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.325976    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-multus-conf-dir\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.326108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326000    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-cnibin\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.326108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326024    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-registration-dir\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.326108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326047    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-kubernetes\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.326108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326080    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-sysctl-d\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.326108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326102    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-sysctl-conf\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326120    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-var-lib-cni-bin\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326138    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-run-multus-certs\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326162    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9h4d\" (UniqueName: \"kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d\") pod \"network-check-target-rk9c4\" (UID: \"f07b94b6-47d7-46af-b886-27ae2a6ebdb3\") " pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326192    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-os-release\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326216    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-var-lib-openvswitch\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326244    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c729200d-5807-4807-a078-990cef3e1bb3-kubelet-config\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326268    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-etc-openvswitch\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326294    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-node-log\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326326    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/df8d0717-00ff-43f0-9b4e-7733475844b2-env-overrides\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326349    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/df8d0717-00ff-43f0-9b4e-7733475844b2-ovnkube-script-lib\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326371    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0-agent-certs\") pod \"konnectivity-agent-6p8r2\" (UID: \"58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0\") " pod="kube-system/konnectivity-agent-6p8r2"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326392    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326422    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-device-dir\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326444    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-systemd\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326467    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r67kz\" (UniqueName: \"kubernetes.io/projected/e09b1d16-8eac-4782-9e73-8a891ab18686-kube-api-access-r67kz\") pod \"node-resolver-w4hf9\" (UID: \"e09b1d16-8eac-4782-9e73-8a891ab18686\") " pod="openshift-dns/node-resolver-w4hf9"
Jul 28 01:04:11.326600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326507    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-run-systemd\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326529    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sklnd\" (UniqueName: \"kubernetes.io/projected/6f874e33-77cd-4288-abd3-77433942b384-kube-api-access-sklnd\") pod \"node-ca-dg6qv\" (UID: \"6f874e33-77cd-4288-abd3-77433942b384\") " pod="openshift-image-registry/node-ca-dg6qv"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326549    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6lp9\" (UniqueName: \"kubernetes.io/projected/5b3d5029-ddde-4549-938d-63c6102bea7b-kube-api-access-v6lp9\") pod \"iptables-alerter-xlb4s\" (UID: \"5b3d5029-ddde-4549-938d-63c6102bea7b\") " pod="openshift-network-operator/iptables-alerter-xlb4s"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326577    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-lib-modules\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326599    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-os-release\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326621    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-multus-daemon-config\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326660    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-kubelet\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326681    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-systemd-units\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326704    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-socket-dir\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326728    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-modprobe-d\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326750    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-var-lib-kubelet\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326780    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-multus-socket-dir-parent\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326807    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/df8d0717-00ff-43f0-9b4e-7733475844b2-ovnkube-config\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326830    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn2ng\" (UniqueName: \"kubernetes.io/projected/f53d5a5f-7a61-468f-8fd6-09912822227d-kube-api-access-cn2ng\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326863    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v698g\" (UniqueName: \"kubernetes.io/projected/3d4fed09-210b-4210-8be1-0edd73941e11-kube-api-access-v698g\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326894    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6f874e33-77cd-4288-abd3-77433942b384-serviceca\") pod \"node-ca-dg6qv\" (UID: \"6f874e33-77cd-4288-abd3-77433942b384\") " pod="openshift-image-registry/node-ca-dg6qv"
Jul 28 01:04:11.327136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326920    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-cni-binary-copy\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326943    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326958    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mppzl\" (UniqueName: \"kubernetes.io/projected/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-kube-api-access-mppzl\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326972    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-run-ovn\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.326987    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327006    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-run\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327024    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-sys\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327037    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e09b1d16-8eac-4782-9e73-8a891ab18686-hosts-file\") pod \"node-resolver-w4hf9\" (UID: \"e09b1d16-8eac-4782-9e73-8a891ab18686\") " pod="openshift-dns/node-resolver-w4hf9"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327068    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-system-cni-dir\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327091    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-run-openvswitch\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327106    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0-konnectivity-ca\") pod \"konnectivity-agent-6p8r2\" (UID: \"58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0\") " pod="kube-system/konnectivity-agent-6p8r2"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327121    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b3d5029-ddde-4549-938d-63c6102bea7b-host-slash\") pod \"iptables-alerter-xlb4s\" (UID: \"5b3d5029-ddde-4549-938d-63c6102bea7b\") " pod="openshift-network-operator/iptables-alerter-xlb4s"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327140    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3d4fed09-210b-4210-8be1-0edd73941e11-etc-tuned\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327175    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-run-netns\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327209    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-multus-cni-dir\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327234    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-var-lib-cni-multus\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.327745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327258    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.328312 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327280    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-slash\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.328312 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327319    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c729200d-5807-4807-a078-990cef3e1bb3-dbus\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:11.328312 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327340    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-system-cni-dir\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.328312 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327362    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-cni-binary-copy\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.328312 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327383    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e09b1d16-8eac-4782-9e73-8a891ab18686-tmp-dir\") pod \"node-resolver-w4hf9\" (UID: \"e09b1d16-8eac-4782-9e73-8a891ab18686\") " pod="openshift-dns/node-resolver-w4hf9"
Jul 28 01:04:11.328312 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327409    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.328312 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327431    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhdl6\" (UniqueName: \"kubernetes.io/projected/df8d0717-00ff-43f0-9b4e-7733475844b2-kube-api-access-bhdl6\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.328312 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327453    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-var-lib-kubelet\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.328312 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327498    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.328312 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.327519    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/df8d0717-00ff-43f0-9b4e-7733475844b2-ovn-node-metrics-cert\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.330944 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.330926    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 01:04:11.352000 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.351985    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bg5vd"
Jul 28 01:04:11.360068 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.360054    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bg5vd"
Jul 28 01:04:11.428045 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428021    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5b3d5029-ddde-4549-938d-63c6102bea7b-iptables-alerter-script\") pod \"iptables-alerter-xlb4s\" (UID: \"5b3d5029-ddde-4549-938d-63c6102bea7b\") " pod="openshift-network-operator/iptables-alerter-xlb4s"
Jul 28 01:04:11.428045 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428049    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-sysconfig\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.428232 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428064    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-host\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.428232 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428077    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-cnibin\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.428232 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428097    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-etc-kubernetes\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.428232 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428123    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.428232 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428135    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-sysconfig\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.428232 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428144    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-cni-bin\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.428232 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428182    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-sys-fs\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.428232 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428203    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-run-ovn-kubernetes\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.428232 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428200    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d4fed09-210b-4210-8be1-0edd73941e11-tmp\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.428232 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428220    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-host\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428248    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-hostroot\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428287    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-sys-fs\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428308    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-cni-netd\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428220    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-cni-bin\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428334    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g7f7\" (UniqueName: \"kubernetes.io/projected/f02709be-e454-4322-b57b-333a63c4093f-kube-api-access-8g7f7\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428359    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f874e33-77cd-4288-abd3-77433942b384-host\") pod \"node-ca-dg6qv\" (UID: \"6f874e33-77cd-4288-abd3-77433942b384\") " pod="openshift-image-registry/node-ca-dg6qv"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428377    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-hostroot\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428386    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-cnibin\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428400    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-run-k8s-cni-cncf-io\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428424    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-run-netns\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428426    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-etc-kubernetes\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428426    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-cni-netd\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428448    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-multus-conf-dir\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428457    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f874e33-77cd-4288-abd3-77433942b384-host\") pod \"node-ca-dg6qv\" (UID: \"6f874e33-77cd-4288-abd3-77433942b384\") " pod="openshift-image-registry/node-ca-dg6qv"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428465    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-run-k8s-cni-cncf-io\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428474    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-cnibin\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428478    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-run-netns\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.428659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428511    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428542    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-registration-dir\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428558    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-cnibin\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428566    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-kubernetes\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428598    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-multus-conf-dir\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428605    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-kubernetes\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428624    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-sysctl-d\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428634    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-registration-dir\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428692    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5b3d5029-ddde-4549-938d-63c6102bea7b-iptables-alerter-script\") pod \"iptables-alerter-xlb4s\" (UID: \"5b3d5029-ddde-4549-938d-63c6102bea7b\") " pod="openshift-network-operator/iptables-alerter-xlb4s"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428725    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-sysctl-conf\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428749    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-var-lib-cni-bin\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428749    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-sysctl-d\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428774    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-run-multus-certs\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428842    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-sysctl-conf\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428840    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-var-lib-cni-bin\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428850    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9h4d\" (UniqueName: \"kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d\") pod \"network-check-target-rk9c4\" (UID: \"f07b94b6-47d7-46af-b886-27ae2a6ebdb3\") " pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428873    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-run-multus-certs\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.429281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428875    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-os-release\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428911    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-var-lib-openvswitch\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428929    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-os-release\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428933    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c729200d-5807-4807-a078-990cef3e1bb3-kubelet-config\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428957    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-etc-openvswitch\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.428991    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-etc-openvswitch\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429006    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c729200d-5807-4807-a078-990cef3e1bb3-kubelet-config\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429017    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-node-log\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429046    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-var-lib-openvswitch\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429089    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/df8d0717-00ff-43f0-9b4e-7733475844b2-env-overrides\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429116    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/df8d0717-00ff-43f0-9b4e-7733475844b2-ovnkube-script-lib\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429133    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0-agent-certs\") pod \"konnectivity-agent-6p8r2\" (UID: \"58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0\") " pod="kube-system/konnectivity-agent-6p8r2"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429148    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429162    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-device-dir\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429186    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-systemd\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429187    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-node-log\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429203    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r67kz\" (UniqueName: \"kubernetes.io/projected/e09b1d16-8eac-4782-9e73-8a891ab18686-kube-api-access-r67kz\") pod \"node-resolver-w4hf9\" (UID: \"e09b1d16-8eac-4782-9e73-8a891ab18686\") " pod="openshift-dns/node-resolver-w4hf9"
Jul 28 01:04:11.429817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429220    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-run-systemd\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429234    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sklnd\" (UniqueName: \"kubernetes.io/projected/6f874e33-77cd-4288-abd3-77433942b384-kube-api-access-sklnd\") pod \"node-ca-dg6qv\" (UID: \"6f874e33-77cd-4288-abd3-77433942b384\") " pod="openshift-image-registry/node-ca-dg6qv"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429250    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6lp9\" (UniqueName: \"kubernetes.io/projected/5b3d5029-ddde-4549-938d-63c6102bea7b-kube-api-access-v6lp9\") pod \"iptables-alerter-xlb4s\" (UID: \"5b3d5029-ddde-4549-938d-63c6102bea7b\") " pod="openshift-network-operator/iptables-alerter-xlb4s"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429271    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-lib-modules\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429286    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-os-release\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429301    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-multus-daemon-config\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429342    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-kubelet\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429363    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-systemd-units\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429378    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-socket-dir\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429392    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-modprobe-d\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429406    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-var-lib-kubelet\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429427    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-multus-socket-dir-parent\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429442    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/df8d0717-00ff-43f0-9b4e-7733475844b2-ovnkube-config\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429458    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn2ng\" (UniqueName: \"kubernetes.io/projected/f53d5a5f-7a61-468f-8fd6-09912822227d-kube-api-access-cn2ng\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429474    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v698g\" (UniqueName: \"kubernetes.io/projected/3d4fed09-210b-4210-8be1-0edd73941e11-kube-api-access-v698g\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429511    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6f874e33-77cd-4288-abd3-77433942b384-serviceca\") pod \"node-ca-dg6qv\" (UID: \"6f874e33-77cd-4288-abd3-77433942b384\") " pod="openshift-image-registry/node-ca-dg6qv"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429526    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-cni-binary-copy\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.430404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429543    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429559    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mppzl\" (UniqueName: \"kubernetes.io/projected/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-kube-api-access-mppzl\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429577    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-run-ovn\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429597    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429610    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-run\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429623    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-sys\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429639    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e09b1d16-8eac-4782-9e73-8a891ab18686-hosts-file\") pod \"node-resolver-w4hf9\" (UID: \"e09b1d16-8eac-4782-9e73-8a891ab18686\") " pod="openshift-dns/node-resolver-w4hf9"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429659    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/df8d0717-00ff-43f0-9b4e-7733475844b2-ovnkube-script-lib\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429665    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-system-cni-dir\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429681    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-run-openvswitch\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429714    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-run-openvswitch\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429753    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-var-lib-kubelet\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429764    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-modprobe-d\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429804    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-multus-socket-dir-parent\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429801    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-device-dir\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429837    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-etc-systemd\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429924    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-lib-modules\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.431204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429981    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-os-release\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430091    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430138    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-run-systemd\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.429659    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/df8d0717-00ff-43f0-9b4e-7733475844b2-env-overrides\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430275    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/df8d0717-00ff-43f0-9b4e-7733475844b2-ovnkube-config\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430353    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0-konnectivity-ca\") pod \"konnectivity-agent-6p8r2\" (UID: \"58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0\") " pod="kube-system/konnectivity-agent-6p8r2"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430384    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b3d5029-ddde-4549-938d-63c6102bea7b-host-slash\") pod \"iptables-alerter-xlb4s\" (UID: \"5b3d5029-ddde-4549-938d-63c6102bea7b\") " pod="openshift-network-operator/iptables-alerter-xlb4s"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430409    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3d4fed09-210b-4210-8be1-0edd73941e11-etc-tuned\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430442    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-run-netns\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430466    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-multus-cni-dir\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430505    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-multus-daemon-config\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430511    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-var-lib-cni-multus\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430539    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430542    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-run-ovn\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430565    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-slash\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430591    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c729200d-5807-4807-a078-990cef3e1bb3-dbus\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430615    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-system-cni-dir\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430639    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-cni-binary-copy\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.431970 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.430650    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430657    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-socket-dir\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430663    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e09b1d16-8eac-4782-9e73-8a891ab18686-tmp-dir\") pod \"node-resolver-w4hf9\" (UID: \"e09b1d16-8eac-4782-9e73-8a891ab18686\") " pod="openshift-dns/node-resolver-w4hf9"
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430690    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430715    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-sys\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.430729    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret podName:c729200d-5807-4807-a078-990cef3e1bb3 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:11.930691909 +0000 UTC m=+2.975585856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret") pod "global-pull-secret-syncer-8k57t" (UID: "c729200d-5807-4807-a078-990cef3e1bb3") : object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430755    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3d4fed09-210b-4210-8be1-0edd73941e11-run\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430779    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e09b1d16-8eac-4782-9e73-8a891ab18686-hosts-file\") pod \"node-resolver-w4hf9\" (UID: \"e09b1d16-8eac-4782-9e73-8a891ab18686\") " pod="openshift-dns/node-resolver-w4hf9"
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430832    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhdl6\" (UniqueName: \"kubernetes.io/projected/df8d0717-00ff-43f0-9b4e-7733475844b2-kube-api-access-bhdl6\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430858    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-var-lib-kubelet\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430884    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430910    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/df8d0717-00ff-43f0-9b4e-7733475844b2-ovn-node-metrics-cert\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430931    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-system-cni-dir\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430934    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-etc-selinux\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430978    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-var-lib-kubelet\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.431042    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:11.432702 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.431083    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs podName:f53d5a5f-7a61-468f-8fd6-09912822227d nodeName:}" failed. No retries permitted until 2026-07-28 01:04:11.931068895 +0000 UTC m=+2.975962845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs") pod "network-metrics-daemon-g8zj6" (UID: "f53d5a5f-7a61-468f-8fd6-09912822227d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431120    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430981    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431201    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfs4m\" (UniqueName: \"kubernetes.io/projected/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-kube-api-access-wfs4m\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431210    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431236    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-log-socket\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431346    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-log-socket\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431431    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f02709be-e454-4322-b57b-333a63c4093f-etc-selinux\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431467    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430978    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6f874e33-77cd-4288-abd3-77433942b384-serviceca\") pod \"node-ca-dg6qv\" (UID: \"6f874e33-77cd-4288-abd3-77433942b384\") " pod="openshift-image-registry/node-ca-dg6qv"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431575    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-system-cni-dir\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431606    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-run-netns\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431673    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c729200d-5807-4807-a078-990cef3e1bb3-dbus\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431646    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-slash\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431738    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-multus-cni-dir\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.430567    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-host-kubelet\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431773    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-host-var-lib-cni-multus\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.433666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431812    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0-konnectivity-ca\") pod \"konnectivity-agent-6p8r2\" (UID: \"58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0\") " pod="kube-system/konnectivity-agent-6p8r2"
Jul 28 01:04:11.434535 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431792    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/df8d0717-00ff-43f0-9b4e-7733475844b2-systemd-units\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.434535 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.431875    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b3d5029-ddde-4549-938d-63c6102bea7b-host-slash\") pod \"iptables-alerter-xlb4s\" (UID: \"5b3d5029-ddde-4549-938d-63c6102bea7b\") " pod="openshift-network-operator/iptables-alerter-xlb4s"
Jul 28 01:04:11.434535 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.432364    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-cni-binary-copy\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.434535 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.432580    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e09b1d16-8eac-4782-9e73-8a891ab18686-tmp-dir\") pod \"node-resolver-w4hf9\" (UID: \"e09b1d16-8eac-4782-9e73-8a891ab18686\") " pod="openshift-dns/node-resolver-w4hf9"
Jul 28 01:04:11.434535 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.432667    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.434535 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.433426    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0-agent-certs\") pod \"konnectivity-agent-6p8r2\" (UID: \"58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0\") " pod="kube-system/konnectivity-agent-6p8r2"
Jul 28 01:04:11.434861 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.434836    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d4fed09-210b-4210-8be1-0edd73941e11-tmp\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.436007 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.435983    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-cni-binary-copy\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.436136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.436104    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3d4fed09-210b-4210-8be1-0edd73941e11-etc-tuned\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.436498 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.436468    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/df8d0717-00ff-43f0-9b4e-7733475844b2-ovn-node-metrics-cert\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.437610 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.437595    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 01:04:11.437610 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.437611    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 01:04:11.437702 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.437621    2565 projected.go:196] Error preparing data for projected volume kube-api-access-d9h4d for pod openshift-network-diagnostics/network-check-target-rk9c4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:11.437702 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.437663    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d podName:f07b94b6-47d7-46af-b886-27ae2a6ebdb3 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:11.937653356 +0000 UTC m=+2.982547340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-d9h4d" (UniqueName: "kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d") pod "network-check-target-rk9c4" (UID: "f07b94b6-47d7-46af-b886-27ae2a6ebdb3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:11.441045 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.441018    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sklnd\" (UniqueName: \"kubernetes.io/projected/6f874e33-77cd-4288-abd3-77433942b384-kube-api-access-sklnd\") pod \"node-ca-dg6qv\" (UID: \"6f874e33-77cd-4288-abd3-77433942b384\") " pod="openshift-image-registry/node-ca-dg6qv"
Jul 28 01:04:11.441183 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.441156    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r67kz\" (UniqueName: \"kubernetes.io/projected/e09b1d16-8eac-4782-9e73-8a891ab18686-kube-api-access-r67kz\") pod \"node-resolver-w4hf9\" (UID: \"e09b1d16-8eac-4782-9e73-8a891ab18686\") " pod="openshift-dns/node-resolver-w4hf9"
Jul 28 01:04:11.441637 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.441606    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6lp9\" (UniqueName: \"kubernetes.io/projected/5b3d5029-ddde-4549-938d-63c6102bea7b-kube-api-access-v6lp9\") pod \"iptables-alerter-xlb4s\" (UID: \"5b3d5029-ddde-4549-938d-63c6102bea7b\") " pod="openshift-network-operator/iptables-alerter-xlb4s"
Jul 28 01:04:11.441728 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.441702    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v698g\" (UniqueName: \"kubernetes.io/projected/3d4fed09-210b-4210-8be1-0edd73941e11-kube-api-access-v698g\") pod \"tuned-4pjgt\" (UID: \"3d4fed09-210b-4210-8be1-0edd73941e11\") " pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.441846 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.441829    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g7f7\" (UniqueName: \"kubernetes.io/projected/f02709be-e454-4322-b57b-333a63c4093f-kube-api-access-8g7f7\") pod \"aws-ebs-csi-driver-node-2wbfv\" (UID: \"f02709be-e454-4322-b57b-333a63c4093f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.442120 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.442098    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhdl6\" (UniqueName: \"kubernetes.io/projected/df8d0717-00ff-43f0-9b4e-7733475844b2-kube-api-access-bhdl6\") pod \"ovnkube-node-5qk5p\" (UID: \"df8d0717-00ff-43f0-9b4e-7733475844b2\") " pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.442269 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.442252    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn2ng\" (UniqueName: \"kubernetes.io/projected/f53d5a5f-7a61-468f-8fd6-09912822227d-kube-api-access-cn2ng\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:11.442540 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.442523    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mppzl\" (UniqueName: \"kubernetes.io/projected/e8d46b27-f0c3-4db4-92fc-1c8d457060fd-kube-api-access-mppzl\") pod \"multus-additional-cni-plugins-xx4zt\" (UID: \"e8d46b27-f0c3-4db4-92fc-1c8d457060fd\") " pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.442855 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.442841    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfs4m\" (UniqueName: \"kubernetes.io/projected/9d8a1856-91c6-49a5-88c7-6e18c5a37a23-kube-api-access-wfs4m\") pod \"multus-74qj2\" (UID: \"9d8a1856-91c6-49a5-88c7-6e18c5a37a23\") " pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.465166 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.465133    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal" event={"ID":"1f5920e338c0b1aa0d84686639273f07","Type":"ContainerStarted","Data":"f9586f69682bf2c6a50351966d2e57c3b49c2ae44e6e837567226d704846fc05"}
Jul 28 01:04:11.466111 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.466092    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-173.ec2.internal" event={"ID":"415ae5ae7d5d3fc074b4ce415469a19e","Type":"ContainerStarted","Data":"ce7082f01d4ec7372cad19afad3f18a0ddc1853e22b4ca453f1b11de90d40331"}
Jul 28 01:04:11.608604 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.608572    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w4hf9"
Jul 28 01:04:11.616674 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.616649    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xx4zt"
Jul 28 01:04:11.622477 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.622255    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:11.626794 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.626776    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6p8r2"
Jul 28 01:04:11.632248 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.632162    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv"
Jul 28 01:04:11.635697 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:11.635666    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d46b27_f0c3_4db4_92fc_1c8d457060fd.slice/crio-827c5543ea29f114c887ea1b4e30703489f21c1fec2010a7d37ba568b929b17d WatchSource:0}: Error finding container 827c5543ea29f114c887ea1b4e30703489f21c1fec2010a7d37ba568b929b17d: Status 404 returned error can't find the container with id 827c5543ea29f114c887ea1b4e30703489f21c1fec2010a7d37ba568b929b17d
Jul 28 01:04:11.637311 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.637286    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4pjgt"
Jul 28 01:04:11.642397 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.642374    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dg6qv"
Jul 28 01:04:11.647850 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.647828    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xlb4s"
Jul 28 01:04:11.649746 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:11.649719    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d4d35b_d49f_4018_bfdb_f3f4a0ad53a0.slice/crio-03432bc3a615b73f774ae66b0f6028fab801b4f80b4d3b63a3563d96e086189e WatchSource:0}: Error finding container 03432bc3a615b73f774ae66b0f6028fab801b4f80b4d3b63a3563d96e086189e: Status 404 returned error can't find the container with id 03432bc3a615b73f774ae66b0f6028fab801b4f80b4d3b63a3563d96e086189e
Jul 28 01:04:11.652502 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.652467    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-74qj2"
Jul 28 01:04:11.653158 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:11.653137    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf02709be_e454_4322_b57b_333a63c4093f.slice/crio-1c70885ca65946f473dd57a42c8b18a3de100fb4962bd50a98816984c4c0e4fd WatchSource:0}: Error finding container 1c70885ca65946f473dd57a42c8b18a3de100fb4962bd50a98816984c4c0e4fd: Status 404 returned error can't find the container with id 1c70885ca65946f473dd57a42c8b18a3de100fb4962bd50a98816984c4c0e4fd
Jul 28 01:04:11.654541 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:11.654520    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8d0717_00ff_43f0_9b4e_7733475844b2.slice/crio-213583829cbe0917ba940188a27fdd4e6e428a5dca7ecbde05cf52ae8bd7926f WatchSource:0}: Error finding container 213583829cbe0917ba940188a27fdd4e6e428a5dca7ecbde05cf52ae8bd7926f: Status 404 returned error can't find the container with id 213583829cbe0917ba940188a27fdd4e6e428a5dca7ecbde05cf52ae8bd7926f
Jul 28 01:04:11.668388 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:11.668364    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f874e33_77cd_4288_abd3_77433942b384.slice/crio-9d5d15c46618e8e124007df5963e319fcd44ac636c9c015a38db95cd79ef505b WatchSource:0}: Error finding container 9d5d15c46618e8e124007df5963e319fcd44ac636c9c015a38db95cd79ef505b: Status 404 returned error can't find the container with id 9d5d15c46618e8e124007df5963e319fcd44ac636c9c015a38db95cd79ef505b
Jul 28 01:04:11.670552 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:11.670524    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d4fed09_210b_4210_8be1_0edd73941e11.slice/crio-8a9d24f3b58ef32895dcd5d9534a2e64fd6c34cd01d46133cd478339e863fd8b WatchSource:0}: Error finding container 8a9d24f3b58ef32895dcd5d9534a2e64fd6c34cd01d46133cd478339e863fd8b: Status 404 returned error can't find the container with id 8a9d24f3b58ef32895dcd5d9534a2e64fd6c34cd01d46133cd478339e863fd8b
Jul 28 01:04:11.672077 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:11.672052    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b3d5029_ddde_4549_938d_63c6102bea7b.slice/crio-14fd708e10e49ca5799f8d03fdfdc4bdec9ecef5458020fa8ed4e2b39c3aece4 WatchSource:0}: Error finding container 14fd708e10e49ca5799f8d03fdfdc4bdec9ecef5458020fa8ed4e2b39c3aece4: Status 404 returned error can't find the container with id 14fd708e10e49ca5799f8d03fdfdc4bdec9ecef5458020fa8ed4e2b39c3aece4
Jul 28 01:04:11.675105 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:11.675074    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d8a1856_91c6_49a5_88c7_6e18c5a37a23.slice/crio-20ae901b4b2aa426c69b90db42e845f714f07ce6d559db8ea7e8f7351d4742c4 WatchSource:0}: Error finding container 20ae901b4b2aa426c69b90db42e845f714f07ce6d559db8ea7e8f7351d4742c4: Status 404 returned error can't find the container with id 20ae901b4b2aa426c69b90db42e845f714f07ce6d559db8ea7e8f7351d4742c4
Jul 28 01:04:11.935428 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.935362    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:11.935428 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:11.935414    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:11.935718 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.935534    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:11.935718 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.935545    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:11.935718 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.935599    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret podName:c729200d-5807-4807-a078-990cef3e1bb3 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:12.935585939 +0000 UTC m=+3.980479885 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret") pod "global-pull-secret-syncer-8k57t" (UID: "c729200d-5807-4807-a078-990cef3e1bb3") : object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:11.935718 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:11.935613    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs podName:f53d5a5f-7a61-468f-8fd6-09912822227d nodeName:}" failed. No retries permitted until 2026-07-28 01:04:12.935606578 +0000 UTC m=+3.980500524 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs") pod "network-metrics-daemon-g8zj6" (UID: "f53d5a5f-7a61-468f-8fd6-09912822227d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:12.036382 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.036182    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9h4d\" (UniqueName: \"kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d\") pod \"network-check-target-rk9c4\" (UID: \"f07b94b6-47d7-46af-b886-27ae2a6ebdb3\") " pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:12.036382 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:12.036334    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 01:04:12.036382 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:12.036359    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 01:04:12.036382 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:12.036368    2565 projected.go:196] Error preparing data for projected volume kube-api-access-d9h4d for pod openshift-network-diagnostics/network-check-target-rk9c4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:12.036712 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:12.036424    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d podName:f07b94b6-47d7-46af-b886-27ae2a6ebdb3 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:13.036402849 +0000 UTC m=+4.081296799 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-d9h4d" (UniqueName: "kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d") pod "network-check-target-rk9c4" (UID: "f07b94b6-47d7-46af-b886-27ae2a6ebdb3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:12.362662 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.362613    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 00:59:11 +0000 UTC" deadline="2028-05-13 03:11:56.94352803 +0000 UTC"
Jul 28 01:04:12.362662 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.362654    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15722h7m44.580879583s"
Jul 28 01:04:12.460824 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.460792    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:12.460997 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:12.460931    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:12.484677 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.484599    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xlb4s" event={"ID":"5b3d5029-ddde-4549-938d-63c6102bea7b","Type":"ContainerStarted","Data":"14fd708e10e49ca5799f8d03fdfdc4bdec9ecef5458020fa8ed4e2b39c3aece4"}
Jul 28 01:04:12.486957 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.486923    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6p8r2" event={"ID":"58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0","Type":"ContainerStarted","Data":"03432bc3a615b73f774ae66b0f6028fab801b4f80b4d3b63a3563d96e086189e"}
Jul 28 01:04:12.489179 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.489153    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4zt" event={"ID":"e8d46b27-f0c3-4db4-92fc-1c8d457060fd","Type":"ContainerStarted","Data":"827c5543ea29f114c887ea1b4e30703489f21c1fec2010a7d37ba568b929b17d"}
Jul 28 01:04:12.491705 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.491677    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w4hf9" event={"ID":"e09b1d16-8eac-4782-9e73-8a891ab18686","Type":"ContainerStarted","Data":"eb074231a856841b0e7cc95b293fb033e4b82f93fe0c8b4f6f48261afc5a05fe"}
Jul 28 01:04:12.493190 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.493167    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dg6qv" event={"ID":"6f874e33-77cd-4288-abd3-77433942b384","Type":"ContainerStarted","Data":"9d5d15c46618e8e124007df5963e319fcd44ac636c9c015a38db95cd79ef505b"}
Jul 28 01:04:12.495123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.495078    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-74qj2" event={"ID":"9d8a1856-91c6-49a5-88c7-6e18c5a37a23","Type":"ContainerStarted","Data":"20ae901b4b2aa426c69b90db42e845f714f07ce6d559db8ea7e8f7351d4742c4"}
Jul 28 01:04:12.496374 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.496351    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4pjgt" event={"ID":"3d4fed09-210b-4210-8be1-0edd73941e11","Type":"ContainerStarted","Data":"8a9d24f3b58ef32895dcd5d9534a2e64fd6c34cd01d46133cd478339e863fd8b"}
Jul 28 01:04:12.500214 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.500115    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv" event={"ID":"f02709be-e454-4322-b57b-333a63c4093f","Type":"ContainerStarted","Data":"1c70885ca65946f473dd57a42c8b18a3de100fb4962bd50a98816984c4c0e4fd"}
Jul 28 01:04:12.502134 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.502112    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p" event={"ID":"df8d0717-00ff-43f0-9b4e-7733475844b2","Type":"ContainerStarted","Data":"213583829cbe0917ba940188a27fdd4e6e428a5dca7ecbde05cf52ae8bd7926f"}
Jul 28 01:04:12.942493 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.942436    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:12.942682 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:12.942555    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:12.942682 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:12.942600    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:12.942682 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:12.942675    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs podName:f53d5a5f-7a61-468f-8fd6-09912822227d nodeName:}" failed. No retries permitted until 2026-07-28 01:04:14.942655187 +0000 UTC m=+5.987549140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs") pod "network-metrics-daemon-g8zj6" (UID: "f53d5a5f-7a61-468f-8fd6-09912822227d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:12.942851 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:12.942758    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:12.942851 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:12.942806    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret podName:c729200d-5807-4807-a078-990cef3e1bb3 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:14.942791285 +0000 UTC m=+5.987685250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret") pod "global-pull-secret-syncer-8k57t" (UID: "c729200d-5807-4807-a078-990cef3e1bb3") : object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:13.043789 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:13.043753    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9h4d\" (UniqueName: \"kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d\") pod \"network-check-target-rk9c4\" (UID: \"f07b94b6-47d7-46af-b886-27ae2a6ebdb3\") " pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:13.043973 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:13.043921    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 01:04:13.043973 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:13.043942    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 01:04:13.043973 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:13.043954    2565 projected.go:196] Error preparing data for projected volume kube-api-access-d9h4d for pod openshift-network-diagnostics/network-check-target-rk9c4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:13.044138 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:13.044009    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d podName:f07b94b6-47d7-46af-b886-27ae2a6ebdb3 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:15.04399161 +0000 UTC m=+6.088885557 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-d9h4d" (UniqueName: "kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d") pod "network-check-target-rk9c4" (UID: "f07b94b6-47d7-46af-b886-27ae2a6ebdb3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:13.363153 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:13.363089    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 00:59:11 +0000 UTC" deadline="2028-01-18 06:20:50.2932666 +0000 UTC"
Jul 28 01:04:13.363153 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:13.363151    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12941h16m36.930119166s"
Jul 28 01:04:13.461665 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:13.461619    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:13.461820 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:13.461745    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:13.463616 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:13.463597    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:13.463736 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:13.463670    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:14.461402 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:14.461255    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:14.461402 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:14.461387    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:14.958925 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:14.958891    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:14.959105 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:14.958977    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:14.959105 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:14.959094    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:14.959243 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:14.959153    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret podName:c729200d-5807-4807-a078-990cef3e1bb3 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:18.959134507 +0000 UTC m=+10.004028457 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret") pod "global-pull-secret-syncer-8k57t" (UID: "c729200d-5807-4807-a078-990cef3e1bb3") : object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:14.959574 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:14.959555    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:14.959665 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:14.959635    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs podName:f53d5a5f-7a61-468f-8fd6-09912822227d nodeName:}" failed. No retries permitted until 2026-07-28 01:04:18.959593493 +0000 UTC m=+10.004487444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs") pod "network-metrics-daemon-g8zj6" (UID: "f53d5a5f-7a61-468f-8fd6-09912822227d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:15.059447 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:15.059411    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9h4d\" (UniqueName: \"kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d\") pod \"network-check-target-rk9c4\" (UID: \"f07b94b6-47d7-46af-b886-27ae2a6ebdb3\") " pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:15.059626 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:15.059609    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 01:04:15.059701 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:15.059627    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 01:04:15.059701 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:15.059640    2565 projected.go:196] Error preparing data for projected volume kube-api-access-d9h4d for pod openshift-network-diagnostics/network-check-target-rk9c4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:15.059701 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:15.059696    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d podName:f07b94b6-47d7-46af-b886-27ae2a6ebdb3 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:19.05967672 +0000 UTC m=+10.104570680 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-d9h4d" (UniqueName: "kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d") pod "network-check-target-rk9c4" (UID: "f07b94b6-47d7-46af-b886-27ae2a6ebdb3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:15.462619 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:15.462585    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:15.462964 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:15.462713    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:15.464350 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:15.463339    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:15.464350 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:15.463430    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:16.461598 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:16.461562    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:16.461785 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:16.461701    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:17.461671 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:17.461593    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:17.462113 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:17.461736    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:17.462236 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:17.462216    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:17.462366 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:17.462345    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:18.461700 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:18.461668    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:18.462148 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:18.461797    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:18.994916 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:18.994878    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:18.995112 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:18.994981    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:18.995112 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:18.995065    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:18.995112 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:18.995085    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:18.995277 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:18.995147    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs podName:f53d5a5f-7a61-468f-8fd6-09912822227d nodeName:}" failed. No retries permitted until 2026-07-28 01:04:26.995126405 +0000 UTC m=+18.040020354 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs") pod "network-metrics-daemon-g8zj6" (UID: "f53d5a5f-7a61-468f-8fd6-09912822227d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:18.995277 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:18.995168    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret podName:c729200d-5807-4807-a078-990cef3e1bb3 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:26.995158694 +0000 UTC m=+18.040052647 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret") pod "global-pull-secret-syncer-8k57t" (UID: "c729200d-5807-4807-a078-990cef3e1bb3") : object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:19.095830 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:19.095794    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9h4d\" (UniqueName: \"kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d\") pod \"network-check-target-rk9c4\" (UID: \"f07b94b6-47d7-46af-b886-27ae2a6ebdb3\") " pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:19.095996 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:19.095964    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 01:04:19.095996 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:19.095982    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 01:04:19.095996 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:19.095994    2565 projected.go:196] Error preparing data for projected volume kube-api-access-d9h4d for pod openshift-network-diagnostics/network-check-target-rk9c4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:19.096139 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:19.096052    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d podName:f07b94b6-47d7-46af-b886-27ae2a6ebdb3 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:27.096034519 +0000 UTC m=+18.140928469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-d9h4d" (UniqueName: "kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d") pod "network-check-target-rk9c4" (UID: "f07b94b6-47d7-46af-b886-27ae2a6ebdb3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:19.463699 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:19.460948    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:19.463699 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:19.461079    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:19.463699 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:19.461151    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:19.463699 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:19.461212    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:20.461167 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:20.461131    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:20.461336 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:20.461273    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:21.460927 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:21.460890    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:21.461313 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:21.461032    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:21.461313 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:21.461086    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:21.461313 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:21.461203    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:22.461297 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:22.461220    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:22.461791 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:22.461352    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:23.461165 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:23.461129    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:23.461351 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:23.461250    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:23.461351 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:23.461286    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:23.461351 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:23.461345    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:24.461224 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:24.461187    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:24.461390 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:24.461337    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:25.461562 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:25.461531    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:25.461562 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:25.461554    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:25.462067 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:25.461646    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:25.462067 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:25.461782    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:26.461129 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:26.461098    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:26.461339 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:26.461199    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:27.056650 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:27.056608    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:27.057123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:27.056674    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:27.057123 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:27.056789    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:27.057123 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:27.056789    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:27.057123 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:27.056857    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs podName:f53d5a5f-7a61-468f-8fd6-09912822227d nodeName:}" failed. No retries permitted until 2026-07-28 01:04:43.056835718 +0000 UTC m=+34.101729666 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs") pod "network-metrics-daemon-g8zj6" (UID: "f53d5a5f-7a61-468f-8fd6-09912822227d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:27.057123 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:27.056875    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret podName:c729200d-5807-4807-a078-990cef3e1bb3 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:43.05686682 +0000 UTC m=+34.101760769 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret") pod "global-pull-secret-syncer-8k57t" (UID: "c729200d-5807-4807-a078-990cef3e1bb3") : object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:27.157193 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:27.157156    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9h4d\" (UniqueName: \"kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d\") pod \"network-check-target-rk9c4\" (UID: \"f07b94b6-47d7-46af-b886-27ae2a6ebdb3\") " pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:27.157389 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:27.157305    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 01:04:27.157389 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:27.157331    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 01:04:27.157389 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:27.157347    2565 projected.go:196] Error preparing data for projected volume kube-api-access-d9h4d for pod openshift-network-diagnostics/network-check-target-rk9c4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:27.157568 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:27.157406    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d podName:f07b94b6-47d7-46af-b886-27ae2a6ebdb3 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:43.157392618 +0000 UTC m=+34.202286564 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-d9h4d" (UniqueName: "kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d") pod "network-check-target-rk9c4" (UID: "f07b94b6-47d7-46af-b886-27ae2a6ebdb3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:27.461867 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:27.461828    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:27.462072 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:27.461883    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:27.462072 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:27.461978    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:27.462182 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:27.462110    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:28.461293 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:28.461261    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:28.461760 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:28.461375    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:29.462105 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:29.462071    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:29.462596 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:29.462180    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:29.462596 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:29.462266    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:29.462596 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:29.462377    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:30.461458 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:30.461422    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:30.461646 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:30.461571    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:31.461596 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:31.461229    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:31.462343 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:31.461645    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:31.462343 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:31.461341    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:31.462343 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:31.462039    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:31.551244 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:31.551209    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-74qj2" event={"ID":"9d8a1856-91c6-49a5-88c7-6e18c5a37a23","Type":"ContainerStarted","Data":"79c9d1451bc09f84f1fd1bb4044842239c64a25708b82d1609316d6ca173026f"}
Jul 28 01:04:31.553047 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:31.553020    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4pjgt" event={"ID":"3d4fed09-210b-4210-8be1-0edd73941e11","Type":"ContainerStarted","Data":"f0ad3b548ccd12e124fb737a2f2ed7ebb96f54c2e393c0ebedc927080172acc0"}
Jul 28 01:04:31.555724 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:31.555706    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:04:31.556062 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:31.556045    2565 generic.go:356] "Generic (PLEG): container finished" podID="df8d0717-00ff-43f0-9b4e-7733475844b2" containerID="44ee702dbfdc0b7bcf2f238a6496c30ef940c6df03efbf09d438446235126480" exitCode=1
Jul 28 01:04:31.556132 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:31.556070    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p" event={"ID":"df8d0717-00ff-43f0-9b4e-7733475844b2","Type":"ContainerStarted","Data":"ed0dc2e29d3cdc2945e5263f01d285cb9e75619ab935ce610d931b0e02f6f7f5"}
Jul 28 01:04:31.556191 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:31.556148    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p" event={"ID":"df8d0717-00ff-43f0-9b4e-7733475844b2","Type":"ContainerStarted","Data":"b5321d2b14c401a11b3f398e94e4ad7ea26aa351737670abe8304c08883191c5"}
Jul 28 01:04:31.556237 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:31.556200    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p" event={"ID":"df8d0717-00ff-43f0-9b4e-7733475844b2","Type":"ContainerDied","Data":"44ee702dbfdc0b7bcf2f238a6496c30ef940c6df03efbf09d438446235126480"}
Jul 28 01:04:31.556312 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:31.556296    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p" event={"ID":"df8d0717-00ff-43f0-9b4e-7733475844b2","Type":"ContainerStarted","Data":"950d64ef21752c20e0ca7ddbd6181f974b99bf8673a8d978e83acd0dacfb3d68"}
Jul 28 01:04:31.557632 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:31.557607    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-173.ec2.internal" event={"ID":"415ae5ae7d5d3fc074b4ce415469a19e","Type":"ContainerStarted","Data":"846d3ee0c97b2ae82f0ca0c0393b866d439a6c9a49f60924ec04b305912f02bb"}
Jul 28 01:04:31.569603 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:31.569550    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-74qj2" podStartSLOduration=2.954218652 podStartE2EDuration="22.56953108s" totalImagesPullingTime="19.615312428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:31.568566535 +0000 UTC m=+22.613460504" watchObservedRunningTime="2026-07-28 01:04:31.56953108 +0000 UTC m=+22.614425045"
Jul 28 01:04:31.587400 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:31.587351    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-173.ec2.internal" podStartSLOduration=21.587337445 podStartE2EDuration="21.587337445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:31.586973936 +0000 UTC m=+22.631867904" watchObservedRunningTime="2026-07-28 01:04:31.587337445 +0000 UTC m=+22.632231413"
Jul 28 01:04:31.607200 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:31.605518    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4pjgt" podStartSLOduration=3.265510038 podStartE2EDuration="22.605472624s" totalImagesPullingTime="19.339962586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:31.603865506 +0000 UTC m=+22.648759475" watchObservedRunningTime="2026-07-28 01:04:31.605472624 +0000 UTC m=+22.650366592"
Jul 28 01:04:32.461229 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.461057    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:32.461379 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:32.461311    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:32.561377 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.561343    2565 generic.go:356] "Generic (PLEG): container finished" podID="e8d46b27-f0c3-4db4-92fc-1c8d457060fd" containerID="9ec90787f814fa5ff3058803649989b338fca6350ceb923eb3bc75b50dc4483b" exitCode=0
Jul 28 01:04:32.561946 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.561441    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4zt" event={"ID":"e8d46b27-f0c3-4db4-92fc-1c8d457060fd","Type":"ContainerDied","Data":"9ec90787f814fa5ff3058803649989b338fca6350ceb923eb3bc75b50dc4483b"}
Jul 28 01:04:32.563336 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.563309    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w4hf9" event={"ID":"e09b1d16-8eac-4782-9e73-8a891ab18686","Type":"ContainerStarted","Data":"6a251cd712d4b6bffa9287f34e9e053e7a50a377ac492a028c78c137778e02cc"}
Jul 28 01:04:32.565046 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.565024    2565 generic.go:356] "Generic (PLEG): container finished" podID="1f5920e338c0b1aa0d84686639273f07" containerID="15fe2a1ad9f8e626bd31b8b5d54fba9a31b94394957c43c833f9501c12177f82" exitCode=0
Jul 28 01:04:32.565149 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.565067    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal" event={"ID":"1f5920e338c0b1aa0d84686639273f07","Type":"ContainerDied","Data":"15fe2a1ad9f8e626bd31b8b5d54fba9a31b94394957c43c833f9501c12177f82"}
Jul 28 01:04:32.566752 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.566721    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dg6qv" event={"ID":"6f874e33-77cd-4288-abd3-77433942b384","Type":"ContainerStarted","Data":"a64fce93f70382c840aa5521493a1eaa5c46b6f5ac2e7391451954c8511ece96"}
Jul 28 01:04:32.568404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.568383    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv" event={"ID":"f02709be-e454-4322-b57b-333a63c4093f","Type":"ContainerStarted","Data":"601260e3c348ee9bdb6804ee647d53fd4533c8e92c443e117dc8690ab463ff67"}
Jul 28 01:04:32.571534 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.571517    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:04:32.571922 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.571900    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p" event={"ID":"df8d0717-00ff-43f0-9b4e-7733475844b2","Type":"ContainerStarted","Data":"0892ceb7e2d1ad531f6c9b9f47f83f6a4ca2350aba23fed9670e8fea66cd4fe4"}
Jul 28 01:04:32.572004 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.571969    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p" event={"ID":"df8d0717-00ff-43f0-9b4e-7733475844b2","Type":"ContainerStarted","Data":"86e24757c2e014b00dd2afdae698266f5b363fa5a755273c82aa2fca81c85594"}
Jul 28 01:04:32.573517 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.573471    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xlb4s" event={"ID":"5b3d5029-ddde-4549-938d-63c6102bea7b","Type":"ContainerStarted","Data":"c42c913cdc24eb656857d110c685d01531a665bc6fc0bf20587e5ef52320fe97"}
Jul 28 01:04:32.575246 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.575225    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6p8r2" event={"ID":"58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0","Type":"ContainerStarted","Data":"ebbafdb9e3f7b3bd80ec8337f3fc7ac81ffdfd602c66b4f6c5b0168ce04b87cd"}
Jul 28 01:04:32.598806 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.598764    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xlb4s" podStartSLOduration=4.358780703 podStartE2EDuration="23.598749253s" totalImagesPullingTime="19.23996855s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:32.598599512 +0000 UTC m=+23.643493461" watchObservedRunningTime="2026-07-28 01:04:32.598749253 +0000 UTC m=+23.643643222"
Jul 28 01:04:32.611988 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.611950    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-w4hf9" podStartSLOduration=4.21996699 podStartE2EDuration="23.611939404s" totalImagesPullingTime="19.391972414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:32.611631522 +0000 UTC m=+23.656525484" watchObservedRunningTime="2026-07-28 01:04:32.611939404 +0000 UTC m=+23.656833371"
Jul 28 01:04:32.625003 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.624967    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6p8r2" podStartSLOduration=4.264620806 podStartE2EDuration="23.624959411s" totalImagesPullingTime="19.360338605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:32.624648063 +0000 UTC m=+23.669542030" watchObservedRunningTime="2026-07-28 01:04:32.624959411 +0000 UTC m=+23.669853378"
Jul 28 01:04:32.650154 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:32.650115    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dg6qv" podStartSLOduration=4.363425116 podStartE2EDuration="23.650102549s" totalImagesPullingTime="19.286677433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:32.649918788 +0000 UTC m=+23.694812756" watchObservedRunningTime="2026-07-28 01:04:32.650102549 +0000 UTC m=+23.694996517"
Jul 28 01:04:33.027236 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:33.027207    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 01:04:33.395023 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:33.394926    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T01:04:33.027226094Z","UUID":"6c397cb4-8e8e-4aba-9e88-e8592b8748fa","Handler":null,"Name":"","Endpoint":""}
Jul 28 01:04:33.397236 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:33.397211    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 01:04:33.397384 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:33.397257    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 01:04:33.461581 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:33.461125    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:33.461581 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:33.461176    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:33.461581 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:33.461254    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:33.461581 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:33.461373    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:33.579192 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:33.579149    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal" event={"ID":"1f5920e338c0b1aa0d84686639273f07","Type":"ContainerStarted","Data":"cb61df9fb24567bb912c8a4507463819bce73c6f7b969b25dc788a810c7fd757"}
Jul 28 01:04:33.581336 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:33.581269    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv" event={"ID":"f02709be-e454-4322-b57b-333a63c4093f","Type":"ContainerStarted","Data":"803d67eb72b0f585e30965042ca61e348aedf06557b7e67a65d0def31dfb18c7"}
Jul 28 01:04:33.593115 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:33.593071    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-173.ec2.internal" podStartSLOduration=23.593057994 podStartE2EDuration="23.593057994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:33.592576601 +0000 UTC m=+24.637470568" watchObservedRunningTime="2026-07-28 01:04:33.593057994 +0000 UTC m=+24.637951962"
Jul 28 01:04:34.461754 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:34.461718    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:34.461929 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:34.461845    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:34.585873 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:34.585802    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv" event={"ID":"f02709be-e454-4322-b57b-333a63c4093f","Type":"ContainerStarted","Data":"660fcba5e5ecf9f51bd108b944c8847fe48ad69b8cdb4d66dfa37d3c2a797f30"}
Jul 28 01:04:34.589611 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:34.589590    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:04:34.590006 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:34.589984    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p" event={"ID":"df8d0717-00ff-43f0-9b4e-7733475844b2","Type":"ContainerStarted","Data":"67c7e2d1f2ded375c16f0606679594f1a60ebde4112d0940160de52505764a89"}
Jul 28 01:04:34.601768 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:34.601726    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2wbfv" podStartSLOduration=3.66804901 podStartE2EDuration="25.601712948s" totalImagesPullingTime="21.933663938s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:34.601234979 +0000 UTC m=+25.646128947" watchObservedRunningTime="2026-07-28 01:04:34.601712948 +0000 UTC m=+25.646606915"
Jul 28 01:04:35.461317 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:35.461241    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:35.461504 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:35.461368    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:35.461504 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:35.461428    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:35.461593 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:35.461530    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:35.822162 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:35.822131    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6p8r2"
Jul 28 01:04:35.822920 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:35.822896    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6p8r2"
Jul 28 01:04:36.461650 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:36.461618    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:36.461819 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:36.461733    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:36.595382 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:36.595353    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6p8r2"
Jul 28 01:04:36.595971 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:36.595949    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6p8r2"
Jul 28 01:04:37.461136 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:37.460922    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:37.461852 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:37.460923    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:37.461852 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:37.461161    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:37.461852 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:37.461257    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:37.599010 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:37.598973    2565 generic.go:356] "Generic (PLEG): container finished" podID="e8d46b27-f0c3-4db4-92fc-1c8d457060fd" containerID="1311b7222981513d63bfbd6f097aae81c3e55bdcb2331295af80d87db48e1aeb" exitCode=0
Jul 28 01:04:37.599161 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:37.599065    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4zt" event={"ID":"e8d46b27-f0c3-4db4-92fc-1c8d457060fd","Type":"ContainerDied","Data":"1311b7222981513d63bfbd6f097aae81c3e55bdcb2331295af80d87db48e1aeb"}
Jul 28 01:04:37.602300 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:37.602285    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:04:37.602705 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:37.602686    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p" event={"ID":"df8d0717-00ff-43f0-9b4e-7733475844b2","Type":"ContainerStarted","Data":"15c8926a4e4ebb20e98466c9ef3e740d4ed54dd364216b15847d777dd79ec1b4"}
Jul 28 01:04:37.602915 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:37.602896    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:37.602990 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:37.602965    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:37.603073 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:37.603059    2565 scope.go:117] "RemoveContainer" containerID="44ee702dbfdc0b7bcf2f238a6496c30ef940c6df03efbf09d438446235126480"
Jul 28 01:04:37.619318 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:37.619299    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:38.461006 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:38.460980    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:38.461221 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:38.461094    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:38.610998 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:38.610830    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:04:38.611367 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:38.611341    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p" event={"ID":"df8d0717-00ff-43f0-9b4e-7733475844b2","Type":"ContainerStarted","Data":"21d55e3f560de7bb5bc404cde76ae38b6c485af079949053eb090583d047bc4d"}
Jul 28 01:04:38.611731 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:38.611617    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:38.628322 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:38.628306    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:04:38.644595 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:38.644554    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p" podStartSLOduration=10.248882165 podStartE2EDuration="29.644534013s" totalImagesPullingTime="19.395651848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:38.642834395 +0000 UTC m=+29.687728378" watchObservedRunningTime="2026-07-28 01:04:38.644534013 +0000 UTC m=+29.689427991"
Jul 28 01:04:39.167317 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:39.167288    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8k57t"]
Jul 28 01:04:39.167508 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:39.167407    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:39.167555 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:39.167526    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:39.170339 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:39.170006    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rk9c4"]
Jul 28 01:04:39.170339 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:39.170109    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:39.170339 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:39.170190    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:39.170687 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:39.170668    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g8zj6"]
Jul 28 01:04:39.170751 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:39.170739    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:39.170841 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:39.170820    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:39.615603 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:39.615567    2565 generic.go:356] "Generic (PLEG): container finished" podID="e8d46b27-f0c3-4db4-92fc-1c8d457060fd" containerID="49c9a4f3c979c946924b2714623744854f6fdaf5018d33927ca1d3a5fd891cc1" exitCode=0
Jul 28 01:04:39.615942 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:39.615660    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4zt" event={"ID":"e8d46b27-f0c3-4db4-92fc-1c8d457060fd","Type":"ContainerDied","Data":"49c9a4f3c979c946924b2714623744854f6fdaf5018d33927ca1d3a5fd891cc1"}
Jul 28 01:04:40.461458 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:40.461432    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:40.461613 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:40.461565    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:41.461395 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:41.461369    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:41.461767 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:41.461467    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:41.461767 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:41.461511    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:41.461767 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:41.461573    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:41.623340 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:41.623306    2565 generic.go:356] "Generic (PLEG): container finished" podID="e8d46b27-f0c3-4db4-92fc-1c8d457060fd" containerID="f0d34c4a5636e23df3b221370c35a9eec2af41c411e8f088c3804bf560396331" exitCode=0
Jul 28 01:04:41.623519 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:41.623350    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4zt" event={"ID":"e8d46b27-f0c3-4db4-92fc-1c8d457060fd","Type":"ContainerDied","Data":"f0d34c4a5636e23df3b221370c35a9eec2af41c411e8f088c3804bf560396331"}
Jul 28 01:04:42.461457 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:42.461369    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:42.461843 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:42.461526    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:04:43.083697 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.083668    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:43.083884 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.083720    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:43.083884 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:43.083820    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:43.083884 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:43.083841    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:43.083884 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:43.083871    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs podName:f53d5a5f-7a61-468f-8fd6-09912822227d nodeName:}" failed. No retries permitted until 2026-07-28 01:05:15.08385678 +0000 UTC m=+66.128750725 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs") pod "network-metrics-daemon-g8zj6" (UID: "f53d5a5f-7a61-468f-8fd6-09912822227d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:43.084067 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:43.083902    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret podName:c729200d-5807-4807-a078-990cef3e1bb3 nodeName:}" failed. No retries permitted until 2026-07-28 01:05:15.083884106 +0000 UTC m=+66.128778055 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret") pod "global-pull-secret-syncer-8k57t" (UID: "c729200d-5807-4807-a078-990cef3e1bb3") : object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:43.184605 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.184572    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9h4d\" (UniqueName: \"kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d\") pod \"network-check-target-rk9c4\" (UID: \"f07b94b6-47d7-46af-b886-27ae2a6ebdb3\") " pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:43.184760 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:43.184725    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 01:04:43.184760 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:43.184745    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 01:04:43.184760 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:43.184757    2565 projected.go:196] Error preparing data for projected volume kube-api-access-d9h4d for pod openshift-network-diagnostics/network-check-target-rk9c4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:43.184906 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:43.184821    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d podName:f07b94b6-47d7-46af-b886-27ae2a6ebdb3 nodeName:}" failed. No retries permitted until 2026-07-28 01:05:15.184801222 +0000 UTC m=+66.229695171 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-d9h4d" (UniqueName: "kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d") pod "network-check-target-rk9c4" (UID: "f07b94b6-47d7-46af-b886-27ae2a6ebdb3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:43.461544 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.461455    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:43.461951 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:43.461607    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rk9c4" podUID="f07b94b6-47d7-46af-b886-27ae2a6ebdb3"
Jul 28 01:04:43.461951 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.461663    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:43.461951 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:43.461781    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8k57t" podUID="c729200d-5807-4807-a078-990cef3e1bb3"
Jul 28 01:04:43.818462 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.818436    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-173.ec2.internal" event="NodeReady"
Jul 28 01:04:43.818621 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.818610    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 01:04:43.850088 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.850065    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78cbc94cd5-2kg4v"]
Jul 28 01:04:43.851877 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.851854    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:43.854731 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.854533    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 01:04:43.854731 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.854536    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bzh4w\""
Jul 28 01:04:43.854891 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.854875    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 01:04:43.855129 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.855111    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 01:04:43.861695 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.861672    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 01:04:43.864851 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.864833    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78cbc94cd5-2kg4v"]
Jul 28 01:04:43.868376 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.868351    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4pjfd"]
Jul 28 01:04:43.870302 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.870287    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nbqk2"]
Jul 28 01:04:43.870437 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.870406    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:43.872162 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.872143    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nbqk2"
Jul 28 01:04:43.873523 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.873357    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 01:04:43.873523 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.873356    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 01:04:43.873523 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.873362    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9pqrg\""
Jul 28 01:04:43.874745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.874723    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 01:04:43.874841 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.874767    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 01:04:43.874903 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.874891    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-w2j6n\""
Jul 28 01:04:43.875161 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.875142    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 01:04:43.879979 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.879953    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nbqk2"]
Jul 28 01:04:43.883635 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.883615    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4pjfd"]
Jul 28 01:04:43.991514 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.991427    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-tmp-dir\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:43.991514 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.991476    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2be7258c-f1ce-42df-8769-90258eda6a2b-cert\") pod \"ingress-canary-nbqk2\" (UID: \"2be7258c-f1ce-42df-8769-90258eda6a2b\") " pod="openshift-ingress-canary/ingress-canary-nbqk2"
Jul 28 01:04:43.991514 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.991511    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvg6z\" (UniqueName: \"kubernetes.io/projected/2be7258c-f1ce-42df-8769-90258eda6a2b-kube-api-access-qvg6z\") pod \"ingress-canary-nbqk2\" (UID: \"2be7258c-f1ce-42df-8769-90258eda6a2b\") " pod="openshift-ingress-canary/ingress-canary-nbqk2"
Jul 28 01:04:43.991750 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.991557    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7b0e5250-f00e-47f7-aa59-a06f76c28c76-image-registry-private-configuration\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:43.991750 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.991608    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7b0e5250-f00e-47f7-aa59-a06f76c28c76-installation-pull-secrets\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:43.991750 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.991649    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-config-volume\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:43.991750 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.991663    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-bound-sa-token\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:43.991750 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.991686    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l48bg\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-kube-api-access-l48bg\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:43.991750 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.991719    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7b0e5250-f00e-47f7-aa59-a06f76c28c76-trusted-ca\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:43.991750 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.991745    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:43.992031 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.991760    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbrjg\" (UniqueName: \"kubernetes.io/projected/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-kube-api-access-lbrjg\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:43.992031 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.991781    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-certificates\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:43.992031 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.991841    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:43.992031 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:43.991872    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7b0e5250-f00e-47f7-aa59-a06f76c28c76-ca-trust-extracted\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.000595 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.000472    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6p8r2_58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0/konnectivity-agent/0.log" limit=1024
Jul 28 01:04:44.084579 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.084543    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6p8r2_58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0/konnectivity-agent/0.log" limit=1024
Jul 28 01:04:44.092523 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.092500    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7b0e5250-f00e-47f7-aa59-a06f76c28c76-trusted-ca\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.092657 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.092540    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:44.092657 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.092569    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbrjg\" (UniqueName: \"kubernetes.io/projected/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-kube-api-access-lbrjg\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:44.092657 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.092593    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-certificates\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.092657 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.092637    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.092833 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.092674    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7b0e5250-f00e-47f7-aa59-a06f76c28c76-ca-trust-extracted\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.092833 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.092703    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-tmp-dir\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:44.092833 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.092738    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2be7258c-f1ce-42df-8769-90258eda6a2b-cert\") pod \"ingress-canary-nbqk2\" (UID: \"2be7258c-f1ce-42df-8769-90258eda6a2b\") " pod="openshift-ingress-canary/ingress-canary-nbqk2"
Jul 28 01:04:44.092833 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.092758    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvg6z\" (UniqueName: \"kubernetes.io/projected/2be7258c-f1ce-42df-8769-90258eda6a2b-kube-api-access-qvg6z\") pod \"ingress-canary-nbqk2\" (UID: \"2be7258c-f1ce-42df-8769-90258eda6a2b\") " pod="openshift-ingress-canary/ingress-canary-nbqk2"
Jul 28 01:04:44.092833 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.092784    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7b0e5250-f00e-47f7-aa59-a06f76c28c76-image-registry-private-configuration\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.092833 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.092820    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7b0e5250-f00e-47f7-aa59-a06f76c28c76-installation-pull-secrets\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.093033 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.092865    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-config-volume\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:44.093033 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.092883    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-bound-sa-token\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.093033 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.092904    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l48bg\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-kube-api-access-l48bg\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.093285 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:44.093256    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 01:04:44.093285 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:44.093282    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78cbc94cd5-2kg4v: secret "image-registry-tls" not found
Jul 28 01:04:44.093427 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:44.093343    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls podName:7b0e5250-f00e-47f7-aa59-a06f76c28c76 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:44.593324305 +0000 UTC m=+35.638218275 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls") pod "image-registry-78cbc94cd5-2kg4v" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76") : secret "image-registry-tls" not found
Jul 28 01:04:44.093819 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.093667    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-tmp-dir\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:44.093819 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.093718    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-certificates\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.093967 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:44.093780    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:04:44.093967 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.093845    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7b0e5250-f00e-47f7-aa59-a06f76c28c76-trusted-ca\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.093967 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:44.093926    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls podName:1d7ebd81-b836-4d70-926d-52fbc1e6eb80 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:44.593907675 +0000 UTC m=+35.638801623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls") pod "dns-default-4pjfd" (UID: "1d7ebd81-b836-4d70-926d-52fbc1e6eb80") : secret "dns-default-metrics-tls" not found
Jul 28 01:04:44.094225 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.094203    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-config-volume\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:44.094284 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.094227    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7b0e5250-f00e-47f7-aa59-a06f76c28c76-ca-trust-extracted\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.098049 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.098022    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7b0e5250-f00e-47f7-aa59-a06f76c28c76-image-registry-private-configuration\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.098334 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.098312    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7b0e5250-f00e-47f7-aa59-a06f76c28c76-installation-pull-secrets\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.098767 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.098746    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2be7258c-f1ce-42df-8769-90258eda6a2b-cert\") pod \"ingress-canary-nbqk2\" (UID: \"2be7258c-f1ce-42df-8769-90258eda6a2b\") " pod="openshift-ingress-canary/ingress-canary-nbqk2"
Jul 28 01:04:44.101465 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.101442    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbrjg\" (UniqueName: \"kubernetes.io/projected/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-kube-api-access-lbrjg\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:44.101580 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.101475    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l48bg\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-kube-api-access-l48bg\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.102639 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.102607    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvg6z\" (UniqueName: \"kubernetes.io/projected/2be7258c-f1ce-42df-8769-90258eda6a2b-kube-api-access-qvg6z\") pod \"ingress-canary-nbqk2\" (UID: \"2be7258c-f1ce-42df-8769-90258eda6a2b\") " pod="openshift-ingress-canary/ingress-canary-nbqk2"
Jul 28 01:04:44.103307 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.103283    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-bound-sa-token\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.189580 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.189541    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nbqk2"
Jul 28 01:04:44.461286 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.461251    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:04:44.464573 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.464547    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 01:04:44.464938 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.464562    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qvhk6\""
Jul 28 01:04:44.597475 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.597444    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:44.597743 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:44.597720    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:04:44.597743 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:44.597739    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:44.597935 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:44.597794    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls podName:1d7ebd81-b836-4d70-926d-52fbc1e6eb80 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:45.597771663 +0000 UTC m=+36.642665631 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls") pod "dns-default-4pjfd" (UID: "1d7ebd81-b836-4d70-926d-52fbc1e6eb80") : secret "dns-default-metrics-tls" not found
Jul 28 01:04:44.597935 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:44.597831    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 01:04:44.597935 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:44.597845    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78cbc94cd5-2kg4v: secret "image-registry-tls" not found
Jul 28 01:04:44.597935 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:44.597883    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls podName:7b0e5250-f00e-47f7-aa59-a06f76c28c76 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:45.597871213 +0000 UTC m=+36.642765163 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls") pod "image-registry-78cbc94cd5-2kg4v" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76") : secret "image-registry-tls" not found
Jul 28 01:04:45.461800 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:45.461766    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:04:45.461959 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:45.461766    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:04:45.464494 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:45.464455    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 01:04:45.465203 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:45.464994    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 01:04:45.465203 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:45.465011    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 01:04:45.465994 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:45.465962    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vxkxb\""
Jul 28 01:04:45.607195 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:45.606925    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:45.607195 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:45.606982    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:45.607195 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:45.607094    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:04:45.607195 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:45.607108    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 01:04:45.607195 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:45.607120    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78cbc94cd5-2kg4v: secret "image-registry-tls" not found
Jul 28 01:04:45.607195 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:45.607168    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls podName:7b0e5250-f00e-47f7-aa59-a06f76c28c76 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:47.607153681 +0000 UTC m=+38.652047627 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls") pod "image-registry-78cbc94cd5-2kg4v" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76") : secret "image-registry-tls" not found
Jul 28 01:04:45.607195 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:45.607180    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls podName:1d7ebd81-b836-4d70-926d-52fbc1e6eb80 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:47.607174606 +0000 UTC m=+38.652068552 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls") pod "dns-default-4pjfd" (UID: "1d7ebd81-b836-4d70-926d-52fbc1e6eb80") : secret "dns-default-metrics-tls" not found
Jul 28 01:04:45.664601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:45.664386    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nbqk2"]
Jul 28 01:04:45.667537 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:04:45.667510    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be7258c_f1ce_42df_8769_90258eda6a2b.slice/crio-d0a87d4c43ed3939ab2ed996f5e3d9efb669880d8b5c1f6eedb12a053e625ec6 WatchSource:0}: Error finding container d0a87d4c43ed3939ab2ed996f5e3d9efb669880d8b5c1f6eedb12a053e625ec6: Status 404 returned error can't find the container with id d0a87d4c43ed3939ab2ed996f5e3d9efb669880d8b5c1f6eedb12a053e625ec6
Jul 28 01:04:46.638256 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:46.638216    2565 generic.go:356] "Generic (PLEG): container finished" podID="e8d46b27-f0c3-4db4-92fc-1c8d457060fd" containerID="44d563e202f32065506543621d8837d7782c24a28eca2fc967338e5a8c2b6069" exitCode=0
Jul 28 01:04:46.639022 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:46.638302    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4zt" event={"ID":"e8d46b27-f0c3-4db4-92fc-1c8d457060fd","Type":"ContainerDied","Data":"44d563e202f32065506543621d8837d7782c24a28eca2fc967338e5a8c2b6069"}
Jul 28 01:04:46.639884 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:46.639857    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nbqk2" event={"ID":"2be7258c-f1ce-42df-8769-90258eda6a2b","Type":"ContainerStarted","Data":"d0a87d4c43ed3939ab2ed996f5e3d9efb669880d8b5c1f6eedb12a053e625ec6"}
Jul 28 01:04:47.621691 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:47.621661    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:47.621895 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:47.621716    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:47.621895 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:47.621823    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:04:47.621895 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:47.621849    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 01:04:47.621895 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:47.621862    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78cbc94cd5-2kg4v: secret "image-registry-tls" not found
Jul 28 01:04:47.621895 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:47.621887    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls podName:1d7ebd81-b836-4d70-926d-52fbc1e6eb80 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:51.621870322 +0000 UTC m=+42.666764284 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls") pod "dns-default-4pjfd" (UID: "1d7ebd81-b836-4d70-926d-52fbc1e6eb80") : secret "dns-default-metrics-tls" not found
Jul 28 01:04:47.622082 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:47.621911    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls podName:7b0e5250-f00e-47f7-aa59-a06f76c28c76 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:51.621895293 +0000 UTC m=+42.666789242 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls") pod "image-registry-78cbc94cd5-2kg4v" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76") : secret "image-registry-tls" not found
Jul 28 01:04:47.646613 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:47.646589    2565 generic.go:356] "Generic (PLEG): container finished" podID="e8d46b27-f0c3-4db4-92fc-1c8d457060fd" containerID="6d7beff3dd471541151885c7dc71d8e9ed7ed6b8049ee33f7f9ee49a9c9b41f7" exitCode=0
Jul 28 01:04:47.646967 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:47.646661    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4zt" event={"ID":"e8d46b27-f0c3-4db4-92fc-1c8d457060fd","Type":"ContainerDied","Data":"6d7beff3dd471541151885c7dc71d8e9ed7ed6b8049ee33f7f9ee49a9c9b41f7"}
Jul 28 01:04:47.982396 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:47.982310    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6p8r2_58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0/konnectivity-agent/0.log" limit=1024
Jul 28 01:04:48.082221 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:48.082183    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6p8r2_58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0/konnectivity-agent/0.log" limit=1024
Jul 28 01:04:48.625986 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:48.625942    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6p8r2_58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0/konnectivity-agent/0.log" limit=1024
Jul 28 01:04:48.652930 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:48.652900    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4zt" event={"ID":"e8d46b27-f0c3-4db4-92fc-1c8d457060fd","Type":"ContainerStarted","Data":"1c16e687c585b858e92e01a386a0c8dc5e5ad27bd4b01f368ff46cf9840419bd"}
Jul 28 01:04:48.654354 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:48.654331    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nbqk2" event={"ID":"2be7258c-f1ce-42df-8769-90258eda6a2b","Type":"ContainerStarted","Data":"7d4a2031b19ce8ec33baf0fab9c185e3bc90eb0c66509762562bba230f862711"}
Jul 28 01:04:48.677133 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:48.677078    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xx4zt" podStartSLOduration=21.668600737 podStartE2EDuration="39.677063177s" totalImagesPullingTime="18.00846244s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:48.676534051 +0000 UTC m=+39.721428019" watchObservedRunningTime="2026-07-28 01:04:48.677063177 +0000 UTC m=+39.721957145"
Jul 28 01:04:48.678923 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:48.678892    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6p8r2_58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0/konnectivity-agent/0.log" limit=1024
Jul 28 01:04:48.690120 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:48.690081    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nbqk2" podStartSLOduration=2.891195335 podStartE2EDuration="5.690067433s" totalImagesPullingTime="2.798872098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:48.689964878 +0000 UTC m=+39.734858846" watchObservedRunningTime="2026-07-28 01:04:48.690067433 +0000 UTC m=+39.734961401"
Jul 28 01:04:48.770550 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:48.770443    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6p8r2_58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0/konnectivity-agent/0.log" limit=1024
Jul 28 01:04:48.822251 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:48.822213    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6p8r2_58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0/konnectivity-agent/0.log" limit=1024
Jul 28 01:04:48.971935 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:48.971896    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6p8r2_58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0/konnectivity-agent/0.log" limit=1024
Jul 28 01:04:49.322258 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:49.322224    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6p8r2_58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0/konnectivity-agent/0.log" limit=1024
Jul 28 01:04:49.424629 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:49.423878    2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6p8r2_58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0/konnectivity-agent/0.log" limit=1024
Jul 28 01:04:51.648468 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:51.648426    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:51.648895 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:51.648508    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:51.648895 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:51.648598    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:04:51.648895 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:51.648658    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls podName:1d7ebd81-b836-4d70-926d-52fbc1e6eb80 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:59.648644083 +0000 UTC m=+50.693538029 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls") pod "dns-default-4pjfd" (UID: "1d7ebd81-b836-4d70-926d-52fbc1e6eb80") : secret "dns-default-metrics-tls" not found
Jul 28 01:04:51.648895 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:51.648665    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 01:04:51.648895 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:51.648686    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78cbc94cd5-2kg4v: secret "image-registry-tls" not found
Jul 28 01:04:51.648895 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:51.648729    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls podName:7b0e5250-f00e-47f7-aa59-a06f76c28c76 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:59.648715428 +0000 UTC m=+50.693609411 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls") pod "image-registry-78cbc94cd5-2kg4v" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76") : secret "image-registry-tls" not found
Jul 28 01:04:59.700729 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:59.700694    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:04:59.701120 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:04:59.700769    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:04:59.701120 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:59.700829    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 01:04:59.701120 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:59.700845    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78cbc94cd5-2kg4v: secret "image-registry-tls" not found
Jul 28 01:04:59.701120 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:59.700853    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:04:59.701120 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:59.700914    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls podName:7b0e5250-f00e-47f7-aa59-a06f76c28c76 nodeName:}" failed. No retries permitted until 2026-07-28 01:05:15.700892783 +0000 UTC m=+66.745786731 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls") pod "image-registry-78cbc94cd5-2kg4v" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76") : secret "image-registry-tls" not found
Jul 28 01:04:59.701120 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:04:59.700928    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls podName:1d7ebd81-b836-4d70-926d-52fbc1e6eb80 nodeName:}" failed. No retries permitted until 2026-07-28 01:05:15.700922149 +0000 UTC m=+66.745816095 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls") pod "dns-default-4pjfd" (UID: "1d7ebd81-b836-4d70-926d-52fbc1e6eb80") : secret "dns-default-metrics-tls" not found
Jul 28 01:05:10.631214 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:10.631184    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5qk5p"
Jul 28 01:05:15.115144 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.115106    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:05:15.115602 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.115176    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:05:15.117592 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.117575    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 01:05:15.117882 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.117866    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 01:05:15.125928 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:05:15.125909    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 01:05:15.125980 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:05:15.125970    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs podName:f53d5a5f-7a61-468f-8fd6-09912822227d nodeName:}" failed. No retries permitted until 2026-07-28 01:06:19.125954488 +0000 UTC m=+130.170848434 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs") pod "network-metrics-daemon-g8zj6" (UID: "f53d5a5f-7a61-468f-8fd6-09912822227d") : secret "metrics-daemon-secret" not found
Jul 28 01:05:15.129340 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.129324    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c729200d-5807-4807-a078-990cef3e1bb3-original-pull-secret\") pod \"global-pull-secret-syncer-8k57t\" (UID: \"c729200d-5807-4807-a078-990cef3e1bb3\") " pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:05:15.172418 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.172382    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8k57t"
Jul 28 01:05:15.216309 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.216278    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9h4d\" (UniqueName: \"kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d\") pod \"network-check-target-rk9c4\" (UID: \"f07b94b6-47d7-46af-b886-27ae2a6ebdb3\") " pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:05:15.219805 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.219572    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 01:05:15.229047 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.228980    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 01:05:15.239458 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.239412    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9h4d\" (UniqueName: \"kubernetes.io/projected/f07b94b6-47d7-46af-b886-27ae2a6ebdb3-kube-api-access-d9h4d\") pod \"network-check-target-rk9c4\" (UID: \"f07b94b6-47d7-46af-b886-27ae2a6ebdb3\") " pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:05:15.292948 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.292916    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8k57t"]
Jul 28 01:05:15.295812 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:05:15.295786    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc729200d_5807_4807_a078_990cef3e1bb3.slice/crio-27154f685638ebb8b4d064a3ac0b455c81efad70adc851505cb7d6376f37f700 WatchSource:0}: Error finding container 27154f685638ebb8b4d064a3ac0b455c81efad70adc851505cb7d6376f37f700: Status 404 returned error can't find the container with id 27154f685638ebb8b4d064a3ac0b455c81efad70adc851505cb7d6376f37f700
Jul 28 01:05:15.481198 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.481124    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vxkxb\""
Jul 28 01:05:15.488066 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.488046    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:05:15.619415 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.619386    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rk9c4"]
Jul 28 01:05:15.622454 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:05:15.622428    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf07b94b6_47d7_46af_b886_27ae2a6ebdb3.slice/crio-9f8686f8fc8711811b78096a5eff69a145084dea3e2f386a1cce30f61ff6e343 WatchSource:0}: Error finding container 9f8686f8fc8711811b78096a5eff69a145084dea3e2f386a1cce30f61ff6e343: Status 404 returned error can't find the container with id 9f8686f8fc8711811b78096a5eff69a145084dea3e2f386a1cce30f61ff6e343
Jul 28 01:05:15.716667 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.716625    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8k57t" event={"ID":"c729200d-5807-4807-a078-990cef3e1bb3","Type":"ContainerStarted","Data":"27154f685638ebb8b4d064a3ac0b455c81efad70adc851505cb7d6376f37f700"}
Jul 28 01:05:15.717650 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.717630    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rk9c4" event={"ID":"f07b94b6-47d7-46af-b886-27ae2a6ebdb3","Type":"ContainerStarted","Data":"9f8686f8fc8711811b78096a5eff69a145084dea3e2f386a1cce30f61ff6e343"}
Jul 28 01:05:15.718955 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.718936    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:05:15.719004 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:15.718973    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:05:15.719085 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:05:15.719071    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:05:15.719135 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:05:15.719126    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls podName:1d7ebd81-b836-4d70-926d-52fbc1e6eb80 nodeName:}" failed. No retries permitted until 2026-07-28 01:05:47.719111582 +0000 UTC m=+98.764005528 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls") pod "dns-default-4pjfd" (UID: "1d7ebd81-b836-4d70-926d-52fbc1e6eb80") : secret "dns-default-metrics-tls" not found
Jul 28 01:05:15.719191 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:05:15.719073    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 01:05:15.719191 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:05:15.719147    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78cbc94cd5-2kg4v: secret "image-registry-tls" not found
Jul 28 01:05:15.719191 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:05:15.719169    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls podName:7b0e5250-f00e-47f7-aa59-a06f76c28c76 nodeName:}" failed. No retries permitted until 2026-07-28 01:05:47.719162689 +0000 UTC m=+98.764056636 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls") pod "image-registry-78cbc94cd5-2kg4v" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76") : secret "image-registry-tls" not found
Jul 28 01:05:19.730863 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:19.730810    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rk9c4" event={"ID":"f07b94b6-47d7-46af-b886-27ae2a6ebdb3","Type":"ContainerStarted","Data":"c7052fe931e6cca934e2c3f4c0a4982e738348bf092c020dc46a0cae801e37b3"}
Jul 28 01:05:19.731326 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:19.730976    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:05:19.745912 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:19.745859    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rk9c4" podStartSLOduration=67.429747466 podStartE2EDuration="1m10.745841075s" totalImagesPullingTime="3.316093609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:05:19.744930781 +0000 UTC m=+70.789824750" watchObservedRunningTime="2026-07-28 01:05:19.745841075 +0000 UTC m=+70.790735023"
Jul 28 01:05:20.734497 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:20.734394    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8k57t" event={"ID":"c729200d-5807-4807-a078-990cef3e1bb3","Type":"ContainerStarted","Data":"87b208091abc2c584ebd18540092d1c27e60966f5257c9367211c3aeaf3a7ccc"}
Jul 28 01:05:20.749466 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:20.749424    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8k57t" podStartSLOduration=66.631746184 podStartE2EDuration="1m11.749410971s" totalImagesPullingTime="5.117664787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:05:20.748936004 +0000 UTC m=+71.793829971" watchObservedRunningTime="2026-07-28 01:05:20.749410971 +0000 UTC m=+71.794304938"
Jul 28 01:05:43.190881 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.190765    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt"]
Jul 28 01:05:43.193674 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.193654    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt"
Jul 28 01:05:43.196035 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.196014    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-fctrr\""
Jul 28 01:05:43.197455 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.197424    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 01:05:43.197455 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.197435    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 01:05:43.197455 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.197439    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 28 01:05:43.197663 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.197522    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 01:05:43.208220 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.208201    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt"]
Jul 28 01:05:43.317723 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.317685    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcdml\" (UniqueName: \"kubernetes.io/projected/07a8ba0a-7ff5-4074-9538-b04c51e568a5-kube-api-access-vcdml\") pod \"managed-serviceaccount-addon-agent-599dc6df8-pstlt\" (UID: \"07a8ba0a-7ff5-4074-9538-b04c51e568a5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt"
Jul 28 01:05:43.317902 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.317746    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07a8ba0a-7ff5-4074-9538-b04c51e568a5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-599dc6df8-pstlt\" (UID: \"07a8ba0a-7ff5-4074-9538-b04c51e568a5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt"
Jul 28 01:05:43.419135 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.419101    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcdml\" (UniqueName: \"kubernetes.io/projected/07a8ba0a-7ff5-4074-9538-b04c51e568a5-kube-api-access-vcdml\") pod \"managed-serviceaccount-addon-agent-599dc6df8-pstlt\" (UID: \"07a8ba0a-7ff5-4074-9538-b04c51e568a5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt"
Jul 28 01:05:43.419259 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.419152    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07a8ba0a-7ff5-4074-9538-b04c51e568a5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-599dc6df8-pstlt\" (UID: \"07a8ba0a-7ff5-4074-9538-b04c51e568a5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt"
Jul 28 01:05:43.421735 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.421714    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/07a8ba0a-7ff5-4074-9538-b04c51e568a5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-599dc6df8-pstlt\" (UID: \"07a8ba0a-7ff5-4074-9538-b04c51e568a5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt"
Jul 28 01:05:43.427333 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.427311    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcdml\" (UniqueName: \"kubernetes.io/projected/07a8ba0a-7ff5-4074-9538-b04c51e568a5-kube-api-access-vcdml\") pod \"managed-serviceaccount-addon-agent-599dc6df8-pstlt\" (UID: \"07a8ba0a-7ff5-4074-9538-b04c51e568a5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt"
Jul 28 01:05:43.511085 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.511023    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt"
Jul 28 01:05:43.629198 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.629136    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt"]
Jul 28 01:05:43.632603 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:05:43.632579    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07a8ba0a_7ff5_4074_9538_b04c51e568a5.slice/crio-6235b123143fc11ebf1621a8d6bc909cfff6b4750de5949f9644f1ccbedfac8e WatchSource:0}: Error finding container 6235b123143fc11ebf1621a8d6bc909cfff6b4750de5949f9644f1ccbedfac8e: Status 404 returned error can't find the container with id 6235b123143fc11ebf1621a8d6bc909cfff6b4750de5949f9644f1ccbedfac8e
Jul 28 01:05:43.791013 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:43.790933    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt" event={"ID":"07a8ba0a-7ff5-4074-9538-b04c51e568a5","Type":"ContainerStarted","Data":"6235b123143fc11ebf1621a8d6bc909cfff6b4750de5949f9644f1ccbedfac8e"}
Jul 28 01:05:46.799401 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:46.799315    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt" event={"ID":"07a8ba0a-7ff5-4074-9538-b04c51e568a5","Type":"ContainerStarted","Data":"0b21e6356d8e03e9f5fac78dd9359139d8fb27a0ede269874ea1cd383235af29"}
Jul 28 01:05:46.814175 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:46.814136    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt" podStartSLOduration=1.032372291 podStartE2EDuration="3.814122936s" totalImagesPullingTime="2.781750645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:05:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:05:46.81304135 +0000 UTC m=+97.857935321" watchObservedRunningTime="2026-07-28 01:05:46.814122936 +0000 UTC m=+97.859016903"
Jul 28 01:05:47.755809 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:47.755772    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:05:47.755809 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:47.755815    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:05:47.756115 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:05:47.755918    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 01:05:47.756115 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:05:47.755929    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78cbc94cd5-2kg4v: secret "image-registry-tls" not found
Jul 28 01:05:47.756115 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:05:47.755929    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:05:47.756115 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:05:47.755988    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls podName:7b0e5250-f00e-47f7-aa59-a06f76c28c76 nodeName:}" failed. No retries permitted until 2026-07-28 01:06:51.755973255 +0000 UTC m=+162.800867201 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls") pod "image-registry-78cbc94cd5-2kg4v" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76") : secret "image-registry-tls" not found
Jul 28 01:05:47.756115 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:05:47.756010    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls podName:1d7ebd81-b836-4d70-926d-52fbc1e6eb80 nodeName:}" failed. No retries permitted until 2026-07-28 01:06:51.755993386 +0000 UTC m=+162.800887352 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls") pod "dns-default-4pjfd" (UID: "1d7ebd81-b836-4d70-926d-52fbc1e6eb80") : secret "dns-default-metrics-tls" not found
Jul 28 01:05:50.737137 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:05:50.737108    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rk9c4"
Jul 28 01:06:19.174610 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:06:19.174571    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:06:19.175004 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:06:19.174730    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 01:06:19.175004 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:06:19.174808    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs podName:f53d5a5f-7a61-468f-8fd6-09912822227d nodeName:}" failed. No retries permitted until 2026-07-28 01:08:21.174791782 +0000 UTC m=+252.219685728 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs") pod "network-metrics-daemon-g8zj6" (UID: "f53d5a5f-7a61-468f-8fd6-09912822227d") : secret "metrics-daemon-secret" not found
Jul 28 01:06:46.864102 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:06:46.864046    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v" podUID="7b0e5250-f00e-47f7-aa59-a06f76c28c76"
Jul 28 01:06:46.883330 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:06:46.883305    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4pjfd" podUID="1d7ebd81-b836-4d70-926d-52fbc1e6eb80"
Jul 28 01:06:46.942186 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:06:46.942163    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:06:46.942295 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:06:46.942167    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:06:47.471824 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:06:47.471790    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:06:51.785419 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:06:51.785372    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:06:51.785419 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:06:51.785424    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:06:51.785864 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:06:51.785543    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:06:51.785864 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:06:51.785632    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls podName:1d7ebd81-b836-4d70-926d-52fbc1e6eb80 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:53.785615298 +0000 UTC m=+284.830509244 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls") pod "dns-default-4pjfd" (UID: "1d7ebd81-b836-4d70-926d-52fbc1e6eb80") : secret "dns-default-metrics-tls" not found
Jul 28 01:06:51.785864 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:06:51.785548    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 01:06:51.785864 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:06:51.785648    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78cbc94cd5-2kg4v: secret "image-registry-tls" not found
Jul 28 01:06:51.785864 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:06:51.785706    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls podName:7b0e5250-f00e-47f7-aa59-a06f76c28c76 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:53.785692149 +0000 UTC m=+284.830586103 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls") pod "image-registry-78cbc94cd5-2kg4v" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76") : secret "image-registry-tls" not found
Jul 28 01:07:00.461710 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:07:00.461610    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:07:47.083326 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:07:47.083284    2565 generic.go:356] "Generic (PLEG): container finished" podID="07a8ba0a-7ff5-4074-9538-b04c51e568a5" containerID="0b21e6356d8e03e9f5fac78dd9359139d8fb27a0ede269874ea1cd383235af29" exitCode=255
Jul 28 01:07:47.083749 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:07:47.083341    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt" event={"ID":"07a8ba0a-7ff5-4074-9538-b04c51e568a5","Type":"ContainerDied","Data":"0b21e6356d8e03e9f5fac78dd9359139d8fb27a0ede269874ea1cd383235af29"}
Jul 28 01:07:47.088923 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:07:47.088905    2565 scope.go:117] "RemoveContainer" containerID="0b21e6356d8e03e9f5fac78dd9359139d8fb27a0ede269874ea1cd383235af29"
Jul 28 01:07:48.088515 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:07:48.087787    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-599dc6df8-pstlt" event={"ID":"07a8ba0a-7ff5-4074-9538-b04c51e568a5","Type":"ContainerStarted","Data":"f3f5d0b400268e4b300bea0dcfcaf5fe08d6bdb96f1a448b0d78e109c00c67a3"}
Jul 28 01:08:10.076863 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.076831    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4"]
Jul 28 01:08:10.078666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.078650    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-8wq8w"]
Jul 28 01:08:10.078788 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.078771    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4"
Jul 28 01:08:10.080270 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.080249    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8wq8w"
Jul 28 01:08:10.083965 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.083942    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 28 01:08:10.083965 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.083956    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:08:10.084123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.083950    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 28 01:08:10.084123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.084030    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 28 01:08:10.085125 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.085108    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-nznjh\""
Jul 28 01:08:10.085200 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.085145    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-mbtlf\""
Jul 28 01:08:10.095530 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.095511    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4"]
Jul 28 01:08:10.099242 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.099213    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-8wq8w"]
Jul 28 01:08:10.164734 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.164702    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"]
Jul 28 01:08:10.166519 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.166471    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-9mrwl"]
Jul 28 01:08:10.166652 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.166590    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:10.168683 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.168663    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:10.171151 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.170978    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 28 01:08:10.171151 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.170992    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 01:08:10.171151 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.170978    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 28 01:08:10.171151 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.171026    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 01:08:10.171151 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.171005    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 28 01:08:10.171151 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.171029    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-bf49c\""
Jul 28 01:08:10.172403 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.172375    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 28 01:08:10.172522 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.172458    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-44dtq\""
Jul 28 01:08:10.172522 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.172518    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 28 01:08:10.172638 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.172518    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:08:10.176630 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.176613    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 28 01:08:10.180470 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.180441    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-9mrwl"]
Jul 28 01:08:10.181324 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.181308    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"]
Jul 28 01:08:10.225913 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.225879    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70870e41-edec-4901-8c63-9fab7dee0173-serving-cert\") pod \"service-ca-operator-5d6c75f84f-zpfq4\" (UID: \"70870e41-edec-4901-8c63-9fab7dee0173\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4"
Jul 28 01:08:10.225913 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.225916    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70870e41-edec-4901-8c63-9fab7dee0173-config\") pod \"service-ca-operator-5d6c75f84f-zpfq4\" (UID: \"70870e41-edec-4901-8c63-9fab7dee0173\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4"
Jul 28 01:08:10.226108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.225988    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrr87\" (UniqueName: \"kubernetes.io/projected/70870e41-edec-4901-8c63-9fab7dee0173-kube-api-access-lrr87\") pod \"service-ca-operator-5d6c75f84f-zpfq4\" (UID: \"70870e41-edec-4901-8c63-9fab7dee0173\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4"
Jul 28 01:08:10.226108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.226016    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxm89\" (UniqueName: \"kubernetes.io/projected/051b966c-1eee-459a-a950-5a2d102f844a-kube-api-access-gxm89\") pod \"network-check-source-7d88cb55d8-8wq8w\" (UID: \"051b966c-1eee-459a-a950-5a2d102f844a\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8wq8w"
Jul 28 01:08:10.326527 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.326453    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2652d05-c03c-4016-a91d-0e7b279aaa74-trusted-ca\") pod \"console-operator-584dcb6b54-9mrwl\" (UID: \"e2652d05-c03c-4016-a91d-0e7b279aaa74\") " pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:10.326723 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.326553    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrr87\" (UniqueName: \"kubernetes.io/projected/70870e41-edec-4901-8c63-9fab7dee0173-kube-api-access-lrr87\") pod \"service-ca-operator-5d6c75f84f-zpfq4\" (UID: \"70870e41-edec-4901-8c63-9fab7dee0173\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4"
Jul 28 01:08:10.326723 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.326585    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct6pg\" (UniqueName: \"kubernetes.io/projected/2da8692b-45f3-435e-b898-ff3f419c5f4a-kube-api-access-ct6pg\") pod \"cluster-monitoring-operator-548b489cb8-gvcqq\" (UID: \"2da8692b-45f3-435e-b898-ff3f419c5f4a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:10.326723 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.326609    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70870e41-edec-4901-8c63-9fab7dee0173-serving-cert\") pod \"service-ca-operator-5d6c75f84f-zpfq4\" (UID: \"70870e41-edec-4901-8c63-9fab7dee0173\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4"
Jul 28 01:08:10.326723 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.326635    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70870e41-edec-4901-8c63-9fab7dee0173-config\") pod \"service-ca-operator-5d6c75f84f-zpfq4\" (UID: \"70870e41-edec-4901-8c63-9fab7dee0173\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4"
Jul 28 01:08:10.326723 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.326656    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e2652d05-c03c-4016-a91d-0e7b279aaa74-serving-cert\") pod \"console-operator-584dcb6b54-9mrwl\" (UID: \"e2652d05-c03c-4016-a91d-0e7b279aaa74\") " pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:10.327017 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.326779    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gvcqq\" (UID: \"2da8692b-45f3-435e-b898-ff3f419c5f4a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:10.327017 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.326819    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6wb5\" (UniqueName: \"kubernetes.io/projected/e2652d05-c03c-4016-a91d-0e7b279aaa74-kube-api-access-g6wb5\") pod \"console-operator-584dcb6b54-9mrwl\" (UID: \"e2652d05-c03c-4016-a91d-0e7b279aaa74\") " pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:10.327017 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.326845    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2da8692b-45f3-435e-b898-ff3f419c5f4a-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-gvcqq\" (UID: \"2da8692b-45f3-435e-b898-ff3f419c5f4a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:10.327017 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.326898    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxm89\" (UniqueName: \"kubernetes.io/projected/051b966c-1eee-459a-a950-5a2d102f844a-kube-api-access-gxm89\") pod \"network-check-source-7d88cb55d8-8wq8w\" (UID: \"051b966c-1eee-459a-a950-5a2d102f844a\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8wq8w"
Jul 28 01:08:10.327017 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.326951    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e2652d05-c03c-4016-a91d-0e7b279aaa74-config\") pod \"console-operator-584dcb6b54-9mrwl\" (UID: \"e2652d05-c03c-4016-a91d-0e7b279aaa74\") " pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:10.327230 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.327200    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/70870e41-edec-4901-8c63-9fab7dee0173-config\") pod \"service-ca-operator-5d6c75f84f-zpfq4\" (UID: \"70870e41-edec-4901-8c63-9fab7dee0173\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4"
Jul 28 01:08:10.329089 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.329061    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/70870e41-edec-4901-8c63-9fab7dee0173-serving-cert\") pod \"service-ca-operator-5d6c75f84f-zpfq4\" (UID: \"70870e41-edec-4901-8c63-9fab7dee0173\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4"
Jul 28 01:08:10.334616 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.334593    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrr87\" (UniqueName: \"kubernetes.io/projected/70870e41-edec-4901-8c63-9fab7dee0173-kube-api-access-lrr87\") pod \"service-ca-operator-5d6c75f84f-zpfq4\" (UID: \"70870e41-edec-4901-8c63-9fab7dee0173\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4"
Jul 28 01:08:10.334875 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.334854    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxm89\" (UniqueName: \"kubernetes.io/projected/051b966c-1eee-459a-a950-5a2d102f844a-kube-api-access-gxm89\") pod \"network-check-source-7d88cb55d8-8wq8w\" (UID: \"051b966c-1eee-459a-a950-5a2d102f844a\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8wq8w"
Jul 28 01:08:10.388091 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.388053    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4"
Jul 28 01:08:10.393201 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.393169    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8wq8w"
Jul 28 01:08:10.428256 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.428214    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e2652d05-c03c-4016-a91d-0e7b279aaa74-serving-cert\") pod \"console-operator-584dcb6b54-9mrwl\" (UID: \"e2652d05-c03c-4016-a91d-0e7b279aaa74\") " pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:10.428408 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.428285    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gvcqq\" (UID: \"2da8692b-45f3-435e-b898-ff3f419c5f4a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:10.428408 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.428322    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6wb5\" (UniqueName: \"kubernetes.io/projected/e2652d05-c03c-4016-a91d-0e7b279aaa74-kube-api-access-g6wb5\") pod \"console-operator-584dcb6b54-9mrwl\" (UID: \"e2652d05-c03c-4016-a91d-0e7b279aaa74\") " pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:10.428408 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.428349    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2da8692b-45f3-435e-b898-ff3f419c5f4a-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-gvcqq\" (UID: \"2da8692b-45f3-435e-b898-ff3f419c5f4a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:10.428408 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.428393    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e2652d05-c03c-4016-a91d-0e7b279aaa74-config\") pod \"console-operator-584dcb6b54-9mrwl\" (UID: \"e2652d05-c03c-4016-a91d-0e7b279aaa74\") " pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:10.428588 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.428437    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2652d05-c03c-4016-a91d-0e7b279aaa74-trusted-ca\") pod \"console-operator-584dcb6b54-9mrwl\" (UID: \"e2652d05-c03c-4016-a91d-0e7b279aaa74\") " pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:10.428588 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.428522    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct6pg\" (UniqueName: \"kubernetes.io/projected/2da8692b-45f3-435e-b898-ff3f419c5f4a-kube-api-access-ct6pg\") pod \"cluster-monitoring-operator-548b489cb8-gvcqq\" (UID: \"2da8692b-45f3-435e-b898-ff3f419c5f4a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:10.428743 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:10.428710    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 01:08:10.428870 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:10.428794    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls podName:2da8692b-45f3-435e-b898-ff3f419c5f4a nodeName:}" failed. No retries permitted until 2026-07-28 01:08:10.928773701 +0000 UTC m=+241.973667661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-gvcqq" (UID: "2da8692b-45f3-435e-b898-ff3f419c5f4a") : secret "cluster-monitoring-operator-tls" not found
Jul 28 01:08:10.429795 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.429520    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e2652d05-c03c-4016-a91d-0e7b279aaa74-config\") pod \"console-operator-584dcb6b54-9mrwl\" (UID: \"e2652d05-c03c-4016-a91d-0e7b279aaa74\") " pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:10.429795 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.429690    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2da8692b-45f3-435e-b898-ff3f419c5f4a-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-gvcqq\" (UID: \"2da8692b-45f3-435e-b898-ff3f419c5f4a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:10.429795 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.429754    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2652d05-c03c-4016-a91d-0e7b279aaa74-trusted-ca\") pod \"console-operator-584dcb6b54-9mrwl\" (UID: \"e2652d05-c03c-4016-a91d-0e7b279aaa74\") " pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:10.431500 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.431447    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e2652d05-c03c-4016-a91d-0e7b279aaa74-serving-cert\") pod \"console-operator-584dcb6b54-9mrwl\" (UID: \"e2652d05-c03c-4016-a91d-0e7b279aaa74\") " pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:10.437877 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.437821    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct6pg\" (UniqueName: \"kubernetes.io/projected/2da8692b-45f3-435e-b898-ff3f419c5f4a-kube-api-access-ct6pg\") pod \"cluster-monitoring-operator-548b489cb8-gvcqq\" (UID: \"2da8692b-45f3-435e-b898-ff3f419c5f4a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:10.438257 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.438236    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6wb5\" (UniqueName: \"kubernetes.io/projected/e2652d05-c03c-4016-a91d-0e7b279aaa74-kube-api-access-g6wb5\") pod \"console-operator-584dcb6b54-9mrwl\" (UID: \"e2652d05-c03c-4016-a91d-0e7b279aaa74\") " pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:10.483128 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.483084    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:10.518108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.518074    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4"]
Jul 28 01:08:10.521054 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:08:10.521022    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70870e41_edec_4901_8c63_9fab7dee0173.slice/crio-ba8fcc65032adf7e3b50be5b3692b055aa6e3d85c3bb5452a3124a82c17693d9 WatchSource:0}: Error finding container ba8fcc65032adf7e3b50be5b3692b055aa6e3d85c3bb5452a3124a82c17693d9: Status 404 returned error can't find the container with id ba8fcc65032adf7e3b50be5b3692b055aa6e3d85c3bb5452a3124a82c17693d9
Jul 28 01:08:10.535097 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.533953    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-8wq8w"]
Jul 28 01:08:10.538191 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:08:10.538159    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod051b966c_1eee_459a_a950_5a2d102f844a.slice/crio-6ff4074a52d67563f2a7c6ce911aee9dd91319ccb18662e36bbbb8d7a4d9210e WatchSource:0}: Error finding container 6ff4074a52d67563f2a7c6ce911aee9dd91319ccb18662e36bbbb8d7a4d9210e: Status 404 returned error can't find the container with id 6ff4074a52d67563f2a7c6ce911aee9dd91319ccb18662e36bbbb8d7a4d9210e
Jul 28 01:08:10.610960 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.610929    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-9mrwl"]
Jul 28 01:08:10.613540 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:08:10.613516    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2652d05_c03c_4016_a91d_0e7b279aaa74.slice/crio-f88af2a4521089267c2818a1e96d2f7b7518b4dbdbdd0970d883948ad2b8c646 WatchSource:0}: Error finding container f88af2a4521089267c2818a1e96d2f7b7518b4dbdbdd0970d883948ad2b8c646: Status 404 returned error can't find the container with id f88af2a4521089267c2818a1e96d2f7b7518b4dbdbdd0970d883948ad2b8c646
Jul 28 01:08:10.933090 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:10.933000    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gvcqq\" (UID: \"2da8692b-45f3-435e-b898-ff3f419c5f4a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:10.933259 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:10.933158    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 01:08:10.933259 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:10.933227    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls podName:2da8692b-45f3-435e-b898-ff3f419c5f4a nodeName:}" failed. No retries permitted until 2026-07-28 01:08:11.933211672 +0000 UTC m=+242.978105621 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-gvcqq" (UID: "2da8692b-45f3-435e-b898-ff3f419c5f4a") : secret "cluster-monitoring-operator-tls" not found
Jul 28 01:08:11.155559 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:11.155517    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl" event={"ID":"e2652d05-c03c-4016-a91d-0e7b279aaa74","Type":"ContainerStarted","Data":"f88af2a4521089267c2818a1e96d2f7b7518b4dbdbdd0970d883948ad2b8c646"}
Jul 28 01:08:11.157378 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:11.157296    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8wq8w" event={"ID":"051b966c-1eee-459a-a950-5a2d102f844a","Type":"ContainerStarted","Data":"f74be0a16820081e9aba9b6962d5001a91c554d6ed2afb1ffd4dae092d7189b9"}
Jul 28 01:08:11.157378 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:11.157347    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8wq8w" event={"ID":"051b966c-1eee-459a-a950-5a2d102f844a","Type":"ContainerStarted","Data":"6ff4074a52d67563f2a7c6ce911aee9dd91319ccb18662e36bbbb8d7a4d9210e"}
Jul 28 01:08:11.158869 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:11.158842    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4" event={"ID":"70870e41-edec-4901-8c63-9fab7dee0173","Type":"ContainerStarted","Data":"ba8fcc65032adf7e3b50be5b3692b055aa6e3d85c3bb5452a3124a82c17693d9"}
Jul 28 01:08:11.173559 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:11.173501    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-8wq8w" podStartSLOduration=1.173464707 podStartE2EDuration="1.173464707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:11.172794958 +0000 UTC m=+242.217688928" watchObservedRunningTime="2026-07-28 01:08:11.173464707 +0000 UTC m=+242.218358677"
Jul 28 01:08:11.942054 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:11.942005    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gvcqq\" (UID: \"2da8692b-45f3-435e-b898-ff3f419c5f4a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:11.942328 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:11.942160    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 01:08:11.942328 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:11.942234    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls podName:2da8692b-45f3-435e-b898-ff3f419c5f4a nodeName:}" failed. No retries permitted until 2026-07-28 01:08:13.942215735 +0000 UTC m=+244.987109685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-gvcqq" (UID: "2da8692b-45f3-435e-b898-ff3f419c5f4a") : secret "cluster-monitoring-operator-tls" not found
Jul 28 01:08:13.958076 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:13.958019    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gvcqq\" (UID: \"2da8692b-45f3-435e-b898-ff3f419c5f4a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:13.958467 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:13.958193    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 01:08:13.958467 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:13.958278    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls podName:2da8692b-45f3-435e-b898-ff3f419c5f4a nodeName:}" failed. No retries permitted until 2026-07-28 01:08:17.958256829 +0000 UTC m=+249.003150780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-gvcqq" (UID: "2da8692b-45f3-435e-b898-ff3f419c5f4a") : secret "cluster-monitoring-operator-tls" not found
Jul 28 01:08:14.169950 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.169924    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/0.log"
Jul 28 01:08:14.170097 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.169966    2565 generic.go:356] "Generic (PLEG): container finished" podID="e2652d05-c03c-4016-a91d-0e7b279aaa74" containerID="745927e1c428306fd131c47970c5f67ec0fdfb55c0a515e75173735649aeac1f" exitCode=255
Jul 28 01:08:14.170097 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.170058    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl" event={"ID":"e2652d05-c03c-4016-a91d-0e7b279aaa74","Type":"ContainerDied","Data":"745927e1c428306fd131c47970c5f67ec0fdfb55c0a515e75173735649aeac1f"}
Jul 28 01:08:14.170270 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.170253    2565 scope.go:117] "RemoveContainer" containerID="745927e1c428306fd131c47970c5f67ec0fdfb55c0a515e75173735649aeac1f"
Jul 28 01:08:14.171622 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.171595    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4" event={"ID":"70870e41-edec-4901-8c63-9fab7dee0173","Type":"ContainerStarted","Data":"55a2315cafb57b9e0c15e2e2ccf0af318d0c73ffb462690d4a9843fb7d83e131"}
Jul 28 01:08:14.201000 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.200960    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4" podStartSLOduration=1.40363751 podStartE2EDuration="4.200944547s" totalImagesPullingTime="2.797307037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:14.200562907 +0000 UTC m=+245.245456874" watchObservedRunningTime="2026-07-28 01:08:14.200944547 +0000 UTC m=+245.245838514"
Jul 28 01:08:14.546902 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.546824    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qq72q"]
Jul 28 01:08:14.548803 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.548787    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:14.551593 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.551567    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 01:08:14.551707 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.551637    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 01:08:14.551707 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.551574    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7gpmd\""
Jul 28 01:08:14.551707 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.551663    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 01:08:14.551834 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.551567    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 01:08:14.559636 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.559616    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qq72q"]
Jul 28 01:08:14.562940 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.562919    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfbb26c6-b5a3-4a63-abae-d63809b5d362-data-volume\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:14.563050 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.562950    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:14.563050 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.562986    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5bj6\" (UniqueName: \"kubernetes.io/projected/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-api-access-g5bj6\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:14.563050 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.563014    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dfbb26c6-b5a3-4a63-abae-d63809b5d362-crio-socket\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:14.563200 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.563102    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:14.664133 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.664095    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfbb26c6-b5a3-4a63-abae-d63809b5d362-data-volume\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:14.664133 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.664130    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:14.664375 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.664160    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5bj6\" (UniqueName: \"kubernetes.io/projected/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-api-access-g5bj6\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:14.664375 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.664178    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dfbb26c6-b5a3-4a63-abae-d63809b5d362-crio-socket\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:14.664375 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.664230    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:14.664375 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:14.664248    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 01:08:14.664375 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:14.664328    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:14.664375 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.664328    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dfbb26c6-b5a3-4a63-abae-d63809b5d362-crio-socket\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:14.664375 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:14.664349    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm podName:dfbb26c6-b5a3-4a63-abae-d63809b5d362 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:15.164328109 +0000 UTC m=+246.209222055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qq72q" (UID: "dfbb26c6-b5a3-4a63-abae-d63809b5d362") : configmap "kube-rbac-proxy" not found
Jul 28 01:08:14.664375 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:14.664368    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls podName:dfbb26c6-b5a3-4a63-abae-d63809b5d362 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:15.164356701 +0000 UTC m=+246.209250648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qq72q" (UID: "dfbb26c6-b5a3-4a63-abae-d63809b5d362") : secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:14.664740 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.664462    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfbb26c6-b5a3-4a63-abae-d63809b5d362-data-volume\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:14.672287 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:14.672263    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5bj6\" (UniqueName: \"kubernetes.io/projected/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-api-access-g5bj6\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:15.168247 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:15.168207    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:15.168651 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:15.168277    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:15.168651 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:15.168329    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 01:08:15.168651 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:15.168371    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:15.168651 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:15.168399    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm podName:dfbb26c6-b5a3-4a63-abae-d63809b5d362 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:16.168382977 +0000 UTC m=+247.213276925 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qq72q" (UID: "dfbb26c6-b5a3-4a63-abae-d63809b5d362") : configmap "kube-rbac-proxy" not found
Jul 28 01:08:15.168651 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:15.168418    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls podName:dfbb26c6-b5a3-4a63-abae-d63809b5d362 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:16.168406237 +0000 UTC m=+247.213300183 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qq72q" (UID: "dfbb26c6-b5a3-4a63-abae-d63809b5d362") : secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:15.176106 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:15.176083    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:08:15.176496 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:15.176463    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/0.log"
Jul 28 01:08:15.176586 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:15.176527    2565 generic.go:356] "Generic (PLEG): container finished" podID="e2652d05-c03c-4016-a91d-0e7b279aaa74" containerID="8183884c504b49c476609f6b85445aef1a920c7c63aa76ac8b8f999fdacfe097" exitCode=255
Jul 28 01:08:15.176643 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:15.176607    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl" event={"ID":"e2652d05-c03c-4016-a91d-0e7b279aaa74","Type":"ContainerDied","Data":"8183884c504b49c476609f6b85445aef1a920c7c63aa76ac8b8f999fdacfe097"}
Jul 28 01:08:15.176688 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:15.176657    2565 scope.go:117] "RemoveContainer" containerID="745927e1c428306fd131c47970c5f67ec0fdfb55c0a515e75173735649aeac1f"
Jul 28 01:08:15.176843 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:15.176830    2565 scope.go:117] "RemoveContainer" containerID="8183884c504b49c476609f6b85445aef1a920c7c63aa76ac8b8f999fdacfe097"
Jul 28 01:08:15.177027 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:15.177007    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-9mrwl_openshift-console-operator(e2652d05-c03c-4016-a91d-0e7b279aaa74)\"" pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl" podUID="e2652d05-c03c-4016-a91d-0e7b279aaa74"
Jul 28 01:08:16.179139 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.179109    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:16.179645 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.179166    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:16.179645 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:16.179230    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 01:08:16.179645 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:16.179271    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:16.179645 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:16.179280    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm podName:dfbb26c6-b5a3-4a63-abae-d63809b5d362 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:18.179267 +0000 UTC m=+249.224160947 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qq72q" (UID: "dfbb26c6-b5a3-4a63-abae-d63809b5d362") : configmap "kube-rbac-proxy" not found
Jul 28 01:08:16.179645 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:16.179326    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls podName:dfbb26c6-b5a3-4a63-abae-d63809b5d362 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:18.179312409 +0000 UTC m=+249.224206355 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qq72q" (UID: "dfbb26c6-b5a3-4a63-abae-d63809b5d362") : secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:16.181221 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.181200    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:08:16.181559 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.181543    2565 scope.go:117] "RemoveContainer" containerID="8183884c504b49c476609f6b85445aef1a920c7c63aa76ac8b8f999fdacfe097"
Jul 28 01:08:16.181697 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:16.181678    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-9mrwl_openshift-console-operator(e2652d05-c03c-4016-a91d-0e7b279aaa74)\"" pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl" podUID="e2652d05-c03c-4016-a91d-0e7b279aaa74"
Jul 28 01:08:16.701797 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.701766    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-ffnnn"]
Jul 28 01:08:16.704304 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.704286    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-ffnnn"
Jul 28 01:08:16.706883 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.706855    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 28 01:08:16.706978 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.706928    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 28 01:08:16.706978 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.706932    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-kbpxp\""
Jul 28 01:08:16.707946 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.707926    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 28 01:08:16.708035 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.707952    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 28 01:08:16.713772 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.713746    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-ffnnn"]
Jul 28 01:08:16.783655 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.783623    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc5fh\" (UniqueName: \"kubernetes.io/projected/7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3-kube-api-access-lc5fh\") pod \"service-ca-658d7b7c78-ffnnn\" (UID: \"7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3\") " pod="openshift-service-ca/service-ca-658d7b7c78-ffnnn"
Jul 28 01:08:16.783825 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.783695    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3-signing-cabundle\") pod \"service-ca-658d7b7c78-ffnnn\" (UID: \"7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3\") " pod="openshift-service-ca/service-ca-658d7b7c78-ffnnn"
Jul 28 01:08:16.783825 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.783719    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3-signing-key\") pod \"service-ca-658d7b7c78-ffnnn\" (UID: \"7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3\") " pod="openshift-service-ca/service-ca-658d7b7c78-ffnnn"
Jul 28 01:08:16.884035 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.884002    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3-signing-cabundle\") pod \"service-ca-658d7b7c78-ffnnn\" (UID: \"7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3\") " pod="openshift-service-ca/service-ca-658d7b7c78-ffnnn"
Jul 28 01:08:16.884035 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.884041    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3-signing-key\") pod \"service-ca-658d7b7c78-ffnnn\" (UID: \"7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3\") " pod="openshift-service-ca/service-ca-658d7b7c78-ffnnn"
Jul 28 01:08:16.884282 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.884261    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc5fh\" (UniqueName: \"kubernetes.io/projected/7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3-kube-api-access-lc5fh\") pod \"service-ca-658d7b7c78-ffnnn\" (UID: \"7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3\") " pod="openshift-service-ca/service-ca-658d7b7c78-ffnnn"
Jul 28 01:08:16.884673 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.884653    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3-signing-cabundle\") pod \"service-ca-658d7b7c78-ffnnn\" (UID: \"7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3\") " pod="openshift-service-ca/service-ca-658d7b7c78-ffnnn"
Jul 28 01:08:16.886466 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.886442    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3-signing-key\") pod \"service-ca-658d7b7c78-ffnnn\" (UID: \"7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3\") " pod="openshift-service-ca/service-ca-658d7b7c78-ffnnn"
Jul 28 01:08:16.892553 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:16.892533    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc5fh\" (UniqueName: \"kubernetes.io/projected/7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3-kube-api-access-lc5fh\") pod \"service-ca-658d7b7c78-ffnnn\" (UID: \"7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3\") " pod="openshift-service-ca/service-ca-658d7b7c78-ffnnn"
Jul 28 01:08:17.013281 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:17.013193    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-ffnnn"
Jul 28 01:08:17.136373 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:17.136338    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-ffnnn"]
Jul 28 01:08:17.138912 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:08:17.138885    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7459b3f7_0b38_4a83_8bf9_f40efbfb3ee3.slice/crio-250a46673f77d25cf53e7d20c59fb3efef1407d4b0181aa489d81add7cb7681c WatchSource:0}: Error finding container 250a46673f77d25cf53e7d20c59fb3efef1407d4b0181aa489d81add7cb7681c: Status 404 returned error can't find the container with id 250a46673f77d25cf53e7d20c59fb3efef1407d4b0181aa489d81add7cb7681c
Jul 28 01:08:17.185017 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:17.184987    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-ffnnn" event={"ID":"7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3","Type":"ContainerStarted","Data":"250a46673f77d25cf53e7d20c59fb3efef1407d4b0181aa489d81add7cb7681c"}
Jul 28 01:08:17.529009 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:17.528983    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w4hf9_e09b1d16-8eac-4782-9e73-8a891ab18686/dns-node-resolver/0.log"
Jul 28 01:08:17.992126 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:17.992089    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gvcqq\" (UID: \"2da8692b-45f3-435e-b898-ff3f419c5f4a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:17.992329 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:17.992268    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 01:08:17.992387 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:17.992346    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls podName:2da8692b-45f3-435e-b898-ff3f419c5f4a nodeName:}" failed. No retries permitted until 2026-07-28 01:08:25.992324499 +0000 UTC m=+257.037218450 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-gvcqq" (UID: "2da8692b-45f3-435e-b898-ff3f419c5f4a") : secret "cluster-monitoring-operator-tls" not found
Jul 28 01:08:18.130711 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:18.130645    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dg6qv_6f874e33-77cd-4288-abd3-77433942b384/node-ca/0.log"
Jul 28 01:08:18.191128 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:18.191090    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-ffnnn" event={"ID":"7459b3f7-0b38-4a83-8bf9-f40efbfb3ee3","Type":"ContainerStarted","Data":"77a2e3d1212fef209a407cee7e48ba9254ad97b1f09bfd570e4a82c0fa099aa0"}
Jul 28 01:08:18.194368 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:18.194337    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:18.194541 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:18.194446    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:18.194541 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:18.194504    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:18.194746 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:18.194575    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls podName:dfbb26c6-b5a3-4a63-abae-d63809b5d362 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:22.194555744 +0000 UTC m=+253.239449696 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qq72q" (UID: "dfbb26c6-b5a3-4a63-abae-d63809b5d362") : secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:18.194746 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:18.194574    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 01:08:18.194746 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:18.194642    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm podName:dfbb26c6-b5a3-4a63-abae-d63809b5d362 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:22.194612082 +0000 UTC m=+253.239506034 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qq72q" (UID: "dfbb26c6-b5a3-4a63-abae-d63809b5d362") : configmap "kube-rbac-proxy" not found
Jul 28 01:08:18.208372 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:18.208311    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-ffnnn" podStartSLOduration=2.208292308 podStartE2EDuration="2.208292308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:18.208184956 +0000 UTC m=+249.253078923" watchObservedRunningTime="2026-07-28 01:08:18.208292308 +0000 UTC m=+249.253186277"
Jul 28 01:08:19.129360 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:19.129315    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nbqk2_2be7258c-f1ce-42df-8769-90258eda6a2b/serve-healthcheck-canary/0.log"
Jul 28 01:08:20.483818 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:20.483791    2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:20.484191 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:20.483850    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:20.484191 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:20.484155    2565 scope.go:117] "RemoveContainer" containerID="8183884c504b49c476609f6b85445aef1a920c7c63aa76ac8b8f999fdacfe097"
Jul 28 01:08:20.484321 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:20.484301    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-9mrwl_openshift-console-operator(e2652d05-c03c-4016-a91d-0e7b279aaa74)\"" pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl" podUID="e2652d05-c03c-4016-a91d-0e7b279aaa74"
Jul 28 01:08:21.219263 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:21.219225    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:08:21.219449 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:21.219384    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 01:08:21.219544 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:21.219460    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs podName:f53d5a5f-7a61-468f-8fd6-09912822227d nodeName:}" failed. No retries permitted until 2026-07-28 01:10:23.219438729 +0000 UTC m=+374.264332674 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs") pod "network-metrics-daemon-g8zj6" (UID: "f53d5a5f-7a61-468f-8fd6-09912822227d") : secret "metrics-daemon-secret" not found
Jul 28 01:08:22.228149 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:22.228115    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:22.228581 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:22.228177    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:22.228581 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:22.228247    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 01:08:22.228581 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:22.228271    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:22.228581 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:22.228302    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm podName:dfbb26c6-b5a3-4a63-abae-d63809b5d362 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:30.22828443 +0000 UTC m=+261.273178380 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qq72q" (UID: "dfbb26c6-b5a3-4a63-abae-d63809b5d362") : configmap "kube-rbac-proxy" not found
Jul 28 01:08:22.228581 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:22.228322    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls podName:dfbb26c6-b5a3-4a63-abae-d63809b5d362 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:30.228309605 +0000 UTC m=+261.273203551 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qq72q" (UID: "dfbb26c6-b5a3-4a63-abae-d63809b5d362") : secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:26.060327 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:26.060291    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gvcqq\" (UID: \"2da8692b-45f3-435e-b898-ff3f419c5f4a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:26.063021 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:26.062994    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2da8692b-45f3-435e-b898-ff3f419c5f4a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-gvcqq\" (UID: \"2da8692b-45f3-435e-b898-ff3f419c5f4a\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:26.077763 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:26.077741    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"
Jul 28 01:08:26.211064 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:26.211038    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq"]
Jul 28 01:08:26.212842 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:08:26.212803    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2da8692b_45f3_435e_b898_ff3f419c5f4a.slice/crio-6394b89eefb832bd746f5f6bb659f4f7f4246992a73f55dd2fe8ebe13a911d9e WatchSource:0}: Error finding container 6394b89eefb832bd746f5f6bb659f4f7f4246992a73f55dd2fe8ebe13a911d9e: Status 404 returned error can't find the container with id 6394b89eefb832bd746f5f6bb659f4f7f4246992a73f55dd2fe8ebe13a911d9e
Jul 28 01:08:27.220773 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:27.220735    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq" event={"ID":"2da8692b-45f3-435e-b898-ff3f419c5f4a","Type":"ContainerStarted","Data":"6394b89eefb832bd746f5f6bb659f4f7f4246992a73f55dd2fe8ebe13a911d9e"}
Jul 28 01:08:29.229102 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:29.229027    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq" event={"ID":"2da8692b-45f3-435e-b898-ff3f419c5f4a","Type":"ContainerStarted","Data":"6793ef287726452bf2ef9ecb901241e8fab789ebac75d9012c7119efa9965124"}
Jul 28 01:08:29.245803 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:29.245758    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq" podStartSLOduration=17.126088586 podStartE2EDuration="19.245744375s" totalImagesPullingTime="2.119655789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:29.243981013 +0000 UTC m=+260.288874981" watchObservedRunningTime="2026-07-28 01:08:29.245744375 +0000 UTC m=+260.290638385"
Jul 28 01:08:30.298716 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:30.298669    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:30.299176 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:30.298895    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:30.299176 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:30.299161    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfbb26c6-b5a3-4a63-abae-d63809b5d362-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:30.301325 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:30.301302    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfbb26c6-b5a3-4a63-abae-d63809b5d362-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qq72q\" (UID: \"dfbb26c6-b5a3-4a63-abae-d63809b5d362\") " pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:30.457261 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:30.457228    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qq72q"
Jul 28 01:08:30.579692 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:30.579668    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qq72q"]
Jul 28 01:08:30.582060 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:08:30.582029    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfbb26c6_b5a3_4a63_abae_d63809b5d362.slice/crio-93768c7777001a115c174ad14a2d6f449e4b4e9bed568e6af3ab659bb8dac92e WatchSource:0}: Error finding container 93768c7777001a115c174ad14a2d6f449e4b4e9bed568e6af3ab659bb8dac92e: Status 404 returned error can't find the container with id 93768c7777001a115c174ad14a2d6f449e4b4e9bed568e6af3ab659bb8dac92e
Jul 28 01:08:31.240793 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:31.240753    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qq72q" event={"ID":"dfbb26c6-b5a3-4a63-abae-d63809b5d362","Type":"ContainerStarted","Data":"93768c7777001a115c174ad14a2d6f449e4b4e9bed568e6af3ab659bb8dac92e"}
Jul 28 01:08:34.462538 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:34.462166    2565 scope.go:117] "RemoveContainer" containerID="8183884c504b49c476609f6b85445aef1a920c7c63aa76ac8b8f999fdacfe097"
Jul 28 01:08:36.260356 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:36.260318    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:08:36.260847 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:36.260429    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl" event={"ID":"e2652d05-c03c-4016-a91d-0e7b279aaa74","Type":"ContainerStarted","Data":"8657c512637ab5f52a59563c8ae93f5384a089daa1c15d5e55b5cf7c9af999fe"}
Jul 28 01:08:36.261014 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:36.260966    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:36.262382 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:36.262355    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qq72q" event={"ID":"dfbb26c6-b5a3-4a63-abae-d63809b5d362","Type":"ContainerStarted","Data":"fe614d399ae09a84a0ce14a2a27f6d447fe0e477518c34680a8b57a76c2e8e00"}
Jul 28 01:08:36.266934 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:36.266891    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl"
Jul 28 01:08:36.278538 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:36.278463    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-9mrwl" podStartSLOduration=23.575634796 podStartE2EDuration="26.278445859s" totalImagesPullingTime="2.702811063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:36.277463155 +0000 UTC m=+267.322357124" watchObservedRunningTime="2026-07-28 01:08:36.278445859 +0000 UTC m=+267.323339828"
Jul 28 01:08:39.275007 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:39.274976    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qq72q" event={"ID":"dfbb26c6-b5a3-4a63-abae-d63809b5d362","Type":"ContainerStarted","Data":"47ae4e4c9303c2f55a6395cb9b83a288d25484e13a683ceb9598726f48de3fea"}
Jul 28 01:08:39.756154 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:39.756120    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-6rl9b"]
Jul 28 01:08:39.759230 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:39.759202    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-6rl9b"
Jul 28 01:08:39.761921 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:39.761896    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-j2v9w\""
Jul 28 01:08:39.762096 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:39.762035    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 01:08:39.762240 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:39.762209    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 01:08:39.773080 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:39.773055    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-6rl9b"]
Jul 28 01:08:39.883883 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:39.883840    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhwh4\" (UniqueName: \"kubernetes.io/projected/5a341e2a-bbb3-4755-9856-c7fcefe35a59-kube-api-access-hhwh4\") pod \"downloads-7c97dc4555-6rl9b\" (UID: \"5a341e2a-bbb3-4755-9856-c7fcefe35a59\") " pod="openshift-console/downloads-7c97dc4555-6rl9b"
Jul 28 01:08:39.984971 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:39.984933    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhwh4\" (UniqueName: \"kubernetes.io/projected/5a341e2a-bbb3-4755-9856-c7fcefe35a59-kube-api-access-hhwh4\") pod \"downloads-7c97dc4555-6rl9b\" (UID: \"5a341e2a-bbb3-4755-9856-c7fcefe35a59\") " pod="openshift-console/downloads-7c97dc4555-6rl9b"
Jul 28 01:08:39.994115 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:39.994077    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhwh4\" (UniqueName: \"kubernetes.io/projected/5a341e2a-bbb3-4755-9856-c7fcefe35a59-kube-api-access-hhwh4\") pod \"downloads-7c97dc4555-6rl9b\" (UID: \"5a341e2a-bbb3-4755-9856-c7fcefe35a59\") " pod="openshift-console/downloads-7c97dc4555-6rl9b"
Jul 28 01:08:40.072573 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:40.072538    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-6rl9b"
Jul 28 01:08:40.220283 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:40.220246    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-6rl9b"]
Jul 28 01:08:40.223956 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:08:40.223924    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a341e2a_bbb3_4755_9856_c7fcefe35a59.slice/crio-35aa932e7de16ad5bac2b1c1892148b9a2f75c5da05cc44c2b6051f13668b6b7 WatchSource:0}: Error finding container 35aa932e7de16ad5bac2b1c1892148b9a2f75c5da05cc44c2b6051f13668b6b7: Status 404 returned error can't find the container with id 35aa932e7de16ad5bac2b1c1892148b9a2f75c5da05cc44c2b6051f13668b6b7
Jul 28 01:08:40.280074 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:40.280026    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-6rl9b" event={"ID":"5a341e2a-bbb3-4755-9856-c7fcefe35a59","Type":"ContainerStarted","Data":"35aa932e7de16ad5bac2b1c1892148b9a2f75c5da05cc44c2b6051f13668b6b7"}
Jul 28 01:08:43.293527 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:43.293461    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qq72q" event={"ID":"dfbb26c6-b5a3-4a63-abae-d63809b5d362","Type":"ContainerStarted","Data":"cc698c2badea892d7fc7057c25d549641cc5088c12966dc618f6392b1a6cb7a6"}
Jul 28 01:08:43.314977 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:43.314916    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qq72q" podStartSLOduration=17.698167316 podStartE2EDuration="29.314892262s" totalImagesPullingTime="11.616724946s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:08:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:43.312553421 +0000 UTC m=+274.357447392" watchObservedRunningTime="2026-07-28 01:08:43.314892262 +0000 UTC m=+274.359786233"
Jul 28 01:08:44.900399 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:44.900364    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8596c5cc5-nqnhg"]
Jul 28 01:08:44.903773 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:44.903746    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:44.907855 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:44.907829    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 01:08:44.908023 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:44.907832    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 01:08:44.908023 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:44.907878    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 01:08:44.908023 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:44.907829    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dhb6l\""
Jul 28 01:08:44.908023 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:44.907831    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 01:08:44.908023 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:44.907829    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 01:08:44.912229 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:44.912206    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8596c5cc5-nqnhg"]
Jul 28 01:08:45.032333 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.032293    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-config\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.032537 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.032344    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-serving-cert\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.032537 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.032468    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8f4h\" (UniqueName: \"kubernetes.io/projected/7123f7db-1a20-4302-9b3f-94b45d2ef12e-kube-api-access-r8f4h\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.032672 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.032567    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-oauth-config\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.032672 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.032599    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-service-ca\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.032770 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.032673    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-oauth-serving-cert\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.133732 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.133686    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8f4h\" (UniqueName: \"kubernetes.io/projected/7123f7db-1a20-4302-9b3f-94b45d2ef12e-kube-api-access-r8f4h\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.133732 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.133743    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-oauth-config\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.133953 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.133759    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-service-ca\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.133953 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.133831    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-oauth-serving-cert\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.133953 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.133883    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-config\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.133953 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.133927    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-serving-cert\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.134672 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.134617    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-config\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.134672 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.134617    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-service-ca\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.134672 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.134628    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-oauth-serving-cert\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.136446 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.136419    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-oauth-config\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.136615 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.136553    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-serving-cert\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.141423 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.141403    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8f4h\" (UniqueName: \"kubernetes.io/projected/7123f7db-1a20-4302-9b3f-94b45d2ef12e-kube-api-access-r8f4h\") pod \"console-8596c5cc5-nqnhg\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") " pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.215738 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.215659    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:45.387128 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:45.387099    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8596c5cc5-nqnhg"]
Jul 28 01:08:45.389292 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:08:45.389251    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7123f7db_1a20_4302_9b3f_94b45d2ef12e.slice/crio-2ba5fc7d4cd2d8f5bbf26269ad12ab1fa34fafdfb8b7948eddbeb133380c0b8c WatchSource:0}: Error finding container 2ba5fc7d4cd2d8f5bbf26269ad12ab1fa34fafdfb8b7948eddbeb133380c0b8c: Status 404 returned error can't find the container with id 2ba5fc7d4cd2d8f5bbf26269ad12ab1fa34fafdfb8b7948eddbeb133380c0b8c
Jul 28 01:08:46.319387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:46.319350    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8596c5cc5-nqnhg" event={"ID":"7123f7db-1a20-4302-9b3f-94b45d2ef12e","Type":"ContainerStarted","Data":"2ba5fc7d4cd2d8f5bbf26269ad12ab1fa34fafdfb8b7948eddbeb133380c0b8c"}
Jul 28 01:08:49.332822 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:49.332792    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:08:49.333299 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:49.332843    2565 generic.go:356] "Generic (PLEG): container finished" podID="2da8692b-45f3-435e-b898-ff3f419c5f4a" containerID="6793ef287726452bf2ef9ecb901241e8fab789ebac75d9012c7119efa9965124" exitCode=2
Jul 28 01:08:49.333299 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:49.332923    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq" event={"ID":"2da8692b-45f3-435e-b898-ff3f419c5f4a","Type":"ContainerDied","Data":"6793ef287726452bf2ef9ecb901241e8fab789ebac75d9012c7119efa9965124"}
Jul 28 01:08:49.333417 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:49.333309    2565 scope.go:117] "RemoveContainer" containerID="6793ef287726452bf2ef9ecb901241e8fab789ebac75d9012c7119efa9965124"
Jul 28 01:08:49.335174 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:49.335141    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8596c5cc5-nqnhg" event={"ID":"7123f7db-1a20-4302-9b3f-94b45d2ef12e","Type":"ContainerStarted","Data":"98eb18e0ffb7b39c867f90e23751fb80621dda517aef9c5ba0fb4eb8ab70225a"}
Jul 28 01:08:49.364395 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:49.364334    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8596c5cc5-nqnhg" podStartSLOduration=1.781064577 podStartE2EDuration="5.364313098s" totalImagesPullingTime="3.583248521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:49.364110177 +0000 UTC m=+280.409004145" watchObservedRunningTime="2026-07-28 01:08:49.364313098 +0000 UTC m=+280.409207069"
Jul 28 01:08:49.942644 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:49.942603    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v" podUID="7b0e5250-f00e-47f7-aa59-a06f76c28c76"
Jul 28 01:08:49.942808 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:49.942644    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4pjfd" podUID="1d7ebd81-b836-4d70-926d-52fbc1e6eb80"
Jul 28 01:08:50.344611 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:50.344577    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:08:50.345047 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:50.344741    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:08:50.345047 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:50.344784    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-gvcqq" event={"ID":"2da8692b-45f3-435e-b898-ff3f419c5f4a","Type":"ContainerStarted","Data":"eaf4e8cd6fb195adc1aa46e3a12cfcabc1cbb28073079161dc14c8adadbe7fa6"}
Jul 28 01:08:50.345047 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:50.344835    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:08:53.012276 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.012207    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"]
Jul 28 01:08:53.017025 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.017004    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.020690 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.020666    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 01:08:53.021135 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.020964    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 01:08:53.021821 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.021635    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 01:08:53.021821 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.021635    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-ttqjr\""
Jul 28 01:08:53.026397 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.025975    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"]
Jul 28 01:08:53.035909 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.035890    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-466qt"]
Jul 28 01:08:53.039472 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.039403    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.042014 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.041985    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 01:08:53.042363 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.042343    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pq5bc\""
Jul 28 01:08:53.042805 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.042549    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 01:08:53.042805 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.042560    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 01:08:53.105602 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.105566    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-metrics-client-ca\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.105760 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.105622    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l568t\" (UniqueName: \"kubernetes.io/projected/5e6c8e5c-a396-42b4-9820-101ec6d764b5-kube-api-access-l568t\") pod \"openshift-state-metrics-b8f49f5c6-245x2\" (UID: \"5e6c8e5c-a396-42b4-9820-101ec6d764b5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.105760 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.105655    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-sys\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.105760 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.105705    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-root\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.105760 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.105739    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-wtmp\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.105972 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.105766    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mq6k\" (UniqueName: \"kubernetes.io/projected/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-kube-api-access-4mq6k\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.105972 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.105813    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-accelerators-collector-config\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.105972 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.105840    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e6c8e5c-a396-42b4-9820-101ec6d764b5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-245x2\" (UID: \"5e6c8e5c-a396-42b4-9820-101ec6d764b5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.105972 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.105865    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e6c8e5c-a396-42b4-9820-101ec6d764b5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-245x2\" (UID: \"5e6c8e5c-a396-42b4-9820-101ec6d764b5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.105972 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.105892    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e6c8e5c-a396-42b4-9820-101ec6d764b5-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-245x2\" (UID: \"5e6c8e5c-a396-42b4-9820-101ec6d764b5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.105972 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.105918    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-tls\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.105972 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.105943    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.106308 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.105986    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-textfile\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207019    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-root\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207081    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-wtmp\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207115    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mq6k\" (UniqueName: \"kubernetes.io/projected/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-kube-api-access-4mq6k\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207159    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-accelerators-collector-config\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207186    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e6c8e5c-a396-42b4-9820-101ec6d764b5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-245x2\" (UID: \"5e6c8e5c-a396-42b4-9820-101ec6d764b5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207211    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e6c8e5c-a396-42b4-9820-101ec6d764b5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-245x2\" (UID: \"5e6c8e5c-a396-42b4-9820-101ec6d764b5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207236    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e6c8e5c-a396-42b4-9820-101ec6d764b5-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-245x2\" (UID: \"5e6c8e5c-a396-42b4-9820-101ec6d764b5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207263    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-tls\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207288    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207328    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-textfile\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207354    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-metrics-client-ca\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207390    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l568t\" (UniqueName: \"kubernetes.io/projected/5e6c8e5c-a396-42b4-9820-101ec6d764b5-kube-api-access-l568t\") pod \"openshift-state-metrics-b8f49f5c6-245x2\" (UID: \"5e6c8e5c-a396-42b4-9820-101ec6d764b5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207419    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-sys\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207534    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-sys\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207586    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-root\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.210147 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.207706    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-wtmp\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.211029 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.208622    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-accelerators-collector-config\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.211029 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:53.208731    2565 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 28 01:08:53.211029 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:53.208784    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e6c8e5c-a396-42b4-9820-101ec6d764b5-openshift-state-metrics-tls podName:5e6c8e5c-a396-42b4-9820-101ec6d764b5 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:53.708767052 +0000 UTC m=+284.753661005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5e6c8e5c-a396-42b4-9820-101ec6d764b5-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-245x2" (UID: "5e6c8e5c-a396-42b4-9820-101ec6d764b5") : secret "openshift-state-metrics-tls" not found
Jul 28 01:08:53.211029 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:53.209384    2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 01:08:53.211029 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:08:53.209442    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-tls podName:dde9b44b-23e1-400e-8ae0-ad8cb7d92759 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:53.709427396 +0000 UTC m=+284.754321349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-tls") pod "node-exporter-466qt" (UID: "dde9b44b-23e1-400e-8ae0-ad8cb7d92759") : secret "node-exporter-tls" not found
Jul 28 01:08:53.211029 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.209532    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-textfile\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.211029 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.210106    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e6c8e5c-a396-42b4-9820-101ec6d764b5-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-245x2\" (UID: \"5e6c8e5c-a396-42b4-9820-101ec6d764b5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.211029 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.210257    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-metrics-client-ca\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.211703 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.211662    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e6c8e5c-a396-42b4-9820-101ec6d764b5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-245x2\" (UID: \"5e6c8e5c-a396-42b4-9820-101ec6d764b5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.214163 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.214132    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.221589 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.221564    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mq6k\" (UniqueName: \"kubernetes.io/projected/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-kube-api-access-4mq6k\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.228043 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.227989    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l568t\" (UniqueName: \"kubernetes.io/projected/5e6c8e5c-a396-42b4-9820-101ec6d764b5-kube-api-access-l568t\") pod \"openshift-state-metrics-b8f49f5c6-245x2\" (UID: \"5e6c8e5c-a396-42b4-9820-101ec6d764b5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.713013 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.712976    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-tls\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.713211 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.713127    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e6c8e5c-a396-42b4-9820-101ec6d764b5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-245x2\" (UID: \"5e6c8e5c-a396-42b4-9820-101ec6d764b5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.715790 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.715763    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dde9b44b-23e1-400e-8ae0-ad8cb7d92759-node-exporter-tls\") pod \"node-exporter-466qt\" (UID: \"dde9b44b-23e1-400e-8ae0-ad8cb7d92759\") " pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.715908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.715823    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e6c8e5c-a396-42b4-9820-101ec6d764b5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-245x2\" (UID: \"5e6c8e5c-a396-42b4-9820-101ec6d764b5\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.814014 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.813980    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:08:53.814196 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.814058    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:08:53.816660 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.816637    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d7ebd81-b836-4d70-926d-52fbc1e6eb80-metrics-tls\") pod \"dns-default-4pjfd\" (UID: \"1d7ebd81-b836-4d70-926d-52fbc1e6eb80\") " pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:08:53.816819 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.816802    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls\") pod \"image-registry-78cbc94cd5-2kg4v\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") " pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:08:53.928807 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.928767    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"
Jul 28 01:08:53.948978 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.948949    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9pqrg\""
Jul 28 01:08:53.949159 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.949035    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bzh4w\""
Jul 28 01:08:53.949983 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.949956    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-466qt"
Jul 28 01:08:53.957049 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.956842    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:08:53.957049 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:53.956859    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:08:55.215999 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:55.215951    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:55.216474 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:55.216032    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:08:55.217611 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:55.217583    2565 patch_prober.go:28] interesting pod/console-8596c5cc5-nqnhg container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.17:8443/health\": dial tcp 10.132.0.17:8443: connect: connection refused" start-of-body=
Jul 28 01:08:55.217726 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:55.217636    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-8596c5cc5-nqnhg" podUID="7123f7db-1a20-4302-9b3f-94b45d2ef12e" containerName="console" probeResult="failure" output="Get \"https://10.132.0.17:8443/health\": dial tcp 10.132.0.17:8443: connect: connection refused"
Jul 28 01:08:57.353706 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.353674    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-79755cbbf5-f99cp"]
Jul 28 01:08:57.359358 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.359335    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.362048 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.362024    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 28 01:08:57.362283 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.362089    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 01:08:57.363462 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.363440    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 28 01:08:57.363576 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.363445    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 28 01:08:57.363576 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.363477    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2r4cj\""
Jul 28 01:08:57.363837 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.363822    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bhgatfk631c5b\""
Jul 28 01:08:57.366879 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.366843    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79755cbbf5-f99cp"]
Jul 28 01:08:57.445679 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.445641    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-metrics-server-audit-profiles\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.445850 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.445703    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74lz7\" (UniqueName: \"kubernetes.io/projected/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-kube-api-access-74lz7\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.445850 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.445779    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.445850 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.445833    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-client-ca-bundle\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.445998 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.445883    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-secret-metrics-server-client-certs\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.445998 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.445953    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-secret-metrics-server-tls\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.445998 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.445989    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-audit-log\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.547384 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.547347    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-secret-metrics-server-tls\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.547594 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.547406    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-audit-log\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.547594 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.547439    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-metrics-server-audit-profiles\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.547594 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.547475    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74lz7\" (UniqueName: \"kubernetes.io/projected/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-kube-api-access-74lz7\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.547594 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.547528    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.547594 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.547571    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-client-ca-bundle\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.547877 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.547748    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-secret-metrics-server-client-certs\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.548317 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.548227    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-audit-log\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.548724 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.548674    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.549065 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.549038    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-metrics-server-audit-profiles\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.550477 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.550426    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-secret-metrics-server-tls\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.550596 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.550509    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-client-ca-bundle\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.550651 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.550614    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-secret-metrics-server-client-certs\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.559780 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.559754    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74lz7\" (UniqueName: \"kubernetes.io/projected/4fb40ff7-f19b-4569-83fe-ffe6f49542dc-kube-api-access-74lz7\") pod \"metrics-server-79755cbbf5-f99cp\" (UID: \"4fb40ff7-f19b-4569-83fe-ffe6f49542dc\") " pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:57.672444 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:57.672373    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:08:58.209785 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.209749    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"]
Jul 28 01:08:58.218680 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.218655    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.222085 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.221762    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 01:08:58.222085 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.221762    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 01:08:58.222085 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.221811    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 01:08:58.222350 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.222110    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-g89zq\""
Jul 28 01:08:58.222350 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.222125    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 01:08:58.222350 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.222147    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 01:08:58.225618 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.225599    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"]
Jul 28 01:08:58.230077 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.230042    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 01:08:58.253474 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.253448    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/95135ee8-17bd-461f-9939-5e7cfb6bfa10-secret-telemeter-client\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.253610 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.253517    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/95135ee8-17bd-461f-9939-5e7cfb6bfa10-telemeter-client-tls\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.253610 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.253573    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95135ee8-17bd-461f-9939-5e7cfb6bfa10-serving-certs-ca-bundle\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.253610 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.253600    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95135ee8-17bd-461f-9939-5e7cfb6bfa10-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.253758 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.253660    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/95135ee8-17bd-461f-9939-5e7cfb6bfa10-federate-client-tls\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.253758 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.253687    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/95135ee8-17bd-461f-9939-5e7cfb6bfa10-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.253838 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.253759    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95135ee8-17bd-461f-9939-5e7cfb6bfa10-metrics-client-ca\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.253838 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.253782    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvbvp\" (UniqueName: \"kubernetes.io/projected/95135ee8-17bd-461f-9939-5e7cfb6bfa10-kube-api-access-gvbvp\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.354562 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.354517    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95135ee8-17bd-461f-9939-5e7cfb6bfa10-metrics-client-ca\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.355026 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.354572    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvbvp\" (UniqueName: \"kubernetes.io/projected/95135ee8-17bd-461f-9939-5e7cfb6bfa10-kube-api-access-gvbvp\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.355026 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.354620    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/95135ee8-17bd-461f-9939-5e7cfb6bfa10-secret-telemeter-client\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.355026 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.354673    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/95135ee8-17bd-461f-9939-5e7cfb6bfa10-telemeter-client-tls\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.355026 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.354721    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95135ee8-17bd-461f-9939-5e7cfb6bfa10-serving-certs-ca-bundle\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.355026 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.354742    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95135ee8-17bd-461f-9939-5e7cfb6bfa10-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.355026 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.354789    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/95135ee8-17bd-461f-9939-5e7cfb6bfa10-federate-client-tls\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.355026 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.354818    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/95135ee8-17bd-461f-9939-5e7cfb6bfa10-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.355967 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.355529    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95135ee8-17bd-461f-9939-5e7cfb6bfa10-serving-certs-ca-bundle\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.355967 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.355602    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95135ee8-17bd-461f-9939-5e7cfb6bfa10-metrics-client-ca\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.356423 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.356398    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95135ee8-17bd-461f-9939-5e7cfb6bfa10-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.357886 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.357866    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/95135ee8-17bd-461f-9939-5e7cfb6bfa10-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.358144 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.358126    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/95135ee8-17bd-461f-9939-5e7cfb6bfa10-federate-client-tls\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.358208 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.358182    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/95135ee8-17bd-461f-9939-5e7cfb6bfa10-secret-telemeter-client\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.358660 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.358640    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/95135ee8-17bd-461f-9939-5e7cfb6bfa10-telemeter-client-tls\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.363248 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.363217    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvbvp\" (UniqueName: \"kubernetes.io/projected/95135ee8-17bd-461f-9939-5e7cfb6bfa10-kube-api-access-gvbvp\") pod \"telemeter-client-7cccb99ffb-kmjvv\" (UID: \"95135ee8-17bd-461f-9939-5e7cfb6bfa10\") " pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:58.531982 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:58.531902    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"
Jul 28 01:08:59.197812 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.197784    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 01:08:59.226357 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.226325    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 01:08:59.226558 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.226469    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.229654 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.229623    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 01:08:59.229789 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.229667    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5iuncbf655mfg\""
Jul 28 01:08:59.229789 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.229713    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v7gcv\""
Jul 28 01:08:59.229789 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.229623    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 01:08:59.229789 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.229734    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 01:08:59.230080 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.230059    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 01:08:59.230206 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.230068    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 01:08:59.231454 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.231015    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 01:08:59.231454 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.231025    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 01:08:59.231454 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.231082    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 01:08:59.231454 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.231171    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 01:08:59.231454 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.231203    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 01:08:59.231454 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.231296    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 01:08:59.231454 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.231023    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 01:08:59.231889 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.231782    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 01:08:59.236882 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.236745    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 01:08:59.264019 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.263984    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264130 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264023    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-web-config\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264130 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264087    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264130 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264118    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264288 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264148    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264288 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264230    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-config\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264284    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264306    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264336    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88b2580a-9d15-45c2-92be-5ab54f563997-config-out\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264357    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264615 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264422    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88b2580a-9d15-45c2-92be-5ab54f563997-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264615 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264476    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264615 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264530    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnkwf\" (UniqueName: \"kubernetes.io/projected/88b2580a-9d15-45c2-92be-5ab54f563997-kube-api-access-vnkwf\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264615 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264557    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264615 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264571    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264615 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264590    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264615 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264604    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264945 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264675    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264945 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264712    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.264945 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.264737    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.365941 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.365907    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.365948    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.365974    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366032    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-config\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366060    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366086    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366121    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88b2580a-9d15-45c2-92be-5ab54f563997-config-out\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366144    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366174    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88b2580a-9d15-45c2-92be-5ab54f563997-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366209    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366235    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnkwf\" (UniqueName: \"kubernetes.io/projected/88b2580a-9d15-45c2-92be-5ab54f563997-kube-api-access-vnkwf\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366269    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366291    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366316    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366340    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.366387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366371    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.367326 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366400    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.367326 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366423    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.367326 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366460    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.367326 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.366782    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.367326 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.367014    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.367326 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.367215    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.367659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.367427    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.368456 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.367931    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.369634 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.369611    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-web-config\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.370162 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.370136    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.370360 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.370338    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-config\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.370643 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.370620    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.370761 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.370739    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.371079 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.371055    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.371327 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.371301    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.371327 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.371318    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.371838 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.371794    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/88b2580a-9d15-45c2-92be-5ab54f563997-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.372123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.372081    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.372222 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.372156    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88b2580a-9d15-45c2-92be-5ab54f563997-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.373085 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.372897    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-web-config\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.373085 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.373038    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88b2580a-9d15-45c2-92be-5ab54f563997-config-out\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.373341 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.373304    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.373690 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.373674    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88b2580a-9d15-45c2-92be-5ab54f563997-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.374983 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.374954    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnkwf\" (UniqueName: \"kubernetes.io/projected/88b2580a-9d15-45c2-92be-5ab54f563997-kube-api-access-vnkwf\") pod \"prometheus-k8s-0\" (UID: \"88b2580a-9d15-45c2-92be-5ab54f563997\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.541287 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.541203    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:08:59.851371 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.851290    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2"]
Jul 28 01:08:59.866405 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:08:59.866383    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4pjfd"]
Jul 28 01:08:59.984808 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:08:59.984776    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e6c8e5c_a396_42b4_9820_101ec6d764b5.slice/crio-79cc2c0a0a26cf51c8f6155ee046a3ff0e4a178a1705959f455d37c14e1b8987 WatchSource:0}: Error finding container 79cc2c0a0a26cf51c8f6155ee046a3ff0e4a178a1705959f455d37c14e1b8987: Status 404 returned error can't find the container with id 79cc2c0a0a26cf51c8f6155ee046a3ff0e4a178a1705959f455d37c14e1b8987
Jul 28 01:08:59.986098 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:08:59.986075    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d7ebd81_b836_4d70_926d_52fbc1e6eb80.slice/crio-12ac8071689e5219c742a33dc0dba9061824197fc700f4e878e35b9138a8c273 WatchSource:0}: Error finding container 12ac8071689e5219c742a33dc0dba9061824197fc700f4e878e35b9138a8c273: Status 404 returned error can't find the container with id 12ac8071689e5219c742a33dc0dba9061824197fc700f4e878e35b9138a8c273
Jul 28 01:09:00.097748 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.097666    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78cbc94cd5-2kg4v"]
Jul 28 01:09:00.103005 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.102980    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv"]
Jul 28 01:09:00.103699 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:09:00.103604    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b0e5250_f00e_47f7_aa59_a06f76c28c76.slice/crio-2fc30d29dcf6f036fbe9ad53d99e07387931161f42a073bd36d7615c3a165d14 WatchSource:0}: Error finding container 2fc30d29dcf6f036fbe9ad53d99e07387931161f42a073bd36d7615c3a165d14: Status 404 returned error can't find the container with id 2fc30d29dcf6f036fbe9ad53d99e07387931161f42a073bd36d7615c3a165d14
Jul 28 01:09:00.111152 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.108817    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79755cbbf5-f99cp"]
Jul 28 01:09:00.115837 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:09:00.115770    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95135ee8_17bd_461f_9939_5e7cfb6bfa10.slice/crio-1601f7f800b177e5fcca261b11a73d6f95de645a5ee3b7aec1f65e2e42726451 WatchSource:0}: Error finding container 1601f7f800b177e5fcca261b11a73d6f95de645a5ee3b7aec1f65e2e42726451: Status 404 returned error can't find the container with id 1601f7f800b177e5fcca261b11a73d6f95de645a5ee3b7aec1f65e2e42726451
Jul 28 01:09:00.117331 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.116658    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 01:09:00.121895 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:09:00.121841    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88b2580a_9d15_45c2_92be_5ab54f563997.slice/crio-352bdfc33c077817112b4ae16097d48d8bd737698f30c5c4580bd01f2f071334 WatchSource:0}: Error finding container 352bdfc33c077817112b4ae16097d48d8bd737698f30c5c4580bd01f2f071334: Status 404 returned error can't find the container with id 352bdfc33c077817112b4ae16097d48d8bd737698f30c5c4580bd01f2f071334
Jul 28 01:09:00.385927 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.385863    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4pjfd" event={"ID":"1d7ebd81-b836-4d70-926d-52fbc1e6eb80","Type":"ContainerStarted","Data":"12ac8071689e5219c742a33dc0dba9061824197fc700f4e878e35b9138a8c273"}
Jul 28 01:09:00.387905 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.387831    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp" event={"ID":"4fb40ff7-f19b-4569-83fe-ffe6f49542dc","Type":"ContainerStarted","Data":"321bffaeadedbd93bff4fa9e8f944d59245737b46fe9011038cb9e00bdff337d"}
Jul 28 01:09:00.390335 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.390267    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv" event={"ID":"95135ee8-17bd-461f-9939-5e7cfb6bfa10","Type":"ContainerStarted","Data":"1601f7f800b177e5fcca261b11a73d6f95de645a5ee3b7aec1f65e2e42726451"}
Jul 28 01:09:00.391790 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.391709    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-466qt" event={"ID":"dde9b44b-23e1-400e-8ae0-ad8cb7d92759","Type":"ContainerStarted","Data":"06bf8cf9747634407dfcb02d6e7b7246ed19b16de77d0e99099e80aba3f57ead"}
Jul 28 01:09:00.394808 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.394664    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2" event={"ID":"5e6c8e5c-a396-42b4-9820-101ec6d764b5","Type":"ContainerStarted","Data":"c6e8139c912e950c31ddc7ab9e9b82d21e1b4a2f9b38720bb2009a0f30fb0cdb"}
Jul 28 01:09:00.394808 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.394725    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2" event={"ID":"5e6c8e5c-a396-42b4-9820-101ec6d764b5","Type":"ContainerStarted","Data":"72d83789843eb0abce4f4c4e4a0cfecd8a7faefa2e5e1cbdb9644d66145dd905"}
Jul 28 01:09:00.394808 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.394757    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2" event={"ID":"5e6c8e5c-a396-42b4-9820-101ec6d764b5","Type":"ContainerStarted","Data":"79cc2c0a0a26cf51c8f6155ee046a3ff0e4a178a1705959f455d37c14e1b8987"}
Jul 28 01:09:00.398734 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.398379    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-6rl9b" event={"ID":"5a341e2a-bbb3-4755-9856-c7fcefe35a59","Type":"ContainerStarted","Data":"2f3ae02c619547d810ea32c2dd3bfe6c965e40819ede940a108876d46546e6af"}
Jul 28 01:09:00.398734 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.398689    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-6rl9b"
Jul 28 01:09:00.403176 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.403031    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v" event={"ID":"7b0e5250-f00e-47f7-aa59-a06f76c28c76","Type":"ContainerStarted","Data":"1f9540ebc012b11906daca24f29dd26647712062344bf9314da0ee55f5ec69c0"}
Jul 28 01:09:00.403176 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.403072    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v" event={"ID":"7b0e5250-f00e-47f7-aa59-a06f76c28c76","Type":"ContainerStarted","Data":"2fc30d29dcf6f036fbe9ad53d99e07387931161f42a073bd36d7615c3a165d14"}
Jul 28 01:09:00.403176 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.403128    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:09:00.403651 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.403605    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-6rl9b"
Jul 28 01:09:00.404695 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.404610    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88b2580a-9d15-45c2-92be-5ab54f563997","Type":"ContainerStarted","Data":"352bdfc33c077817112b4ae16097d48d8bd737698f30c5c4580bd01f2f071334"}
Jul 28 01:09:00.437897 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.436816    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-6rl9b" podStartSLOduration=1.618294237 podStartE2EDuration="21.436793597s" totalImagesPullingTime="19.81849936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:09:00.418105478 +0000 UTC m=+291.462999447" watchObservedRunningTime="2026-07-28 01:09:00.436793597 +0000 UTC m=+291.481687570"
Jul 28 01:09:00.455497 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:00.454509    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v" podStartSLOduration=290.454471696 podStartE2EDuration="4m50.454471696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:09:00.437774439 +0000 UTC m=+291.482668409" watchObservedRunningTime="2026-07-28 01:09:00.454471696 +0000 UTC m=+291.499365667"
Jul 28 01:09:01.417562 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:01.416635    2565 generic.go:356] "Generic (PLEG): container finished" podID="dde9b44b-23e1-400e-8ae0-ad8cb7d92759" containerID="50c0860a9b81697a9cf1372342eba5f20b15aa1df984bee9095037a3741093a5" exitCode=0
Jul 28 01:09:01.417562 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:01.416880    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-466qt" event={"ID":"dde9b44b-23e1-400e-8ae0-ad8cb7d92759","Type":"ContainerDied","Data":"50c0860a9b81697a9cf1372342eba5f20b15aa1df984bee9095037a3741093a5"}
Jul 28 01:09:01.514772 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:01.513962    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8596c5cc5-nqnhg"]
Jul 28 01:09:02.430383 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:02.430287    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-466qt" event={"ID":"dde9b44b-23e1-400e-8ae0-ad8cb7d92759","Type":"ContainerStarted","Data":"29a4c814b704b2a4b524c87c8c4c92b7c86bb9742c68a54a2481e9a932a1cb12"}
Jul 28 01:09:02.430383 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:02.430347    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-466qt" event={"ID":"dde9b44b-23e1-400e-8ae0-ad8cb7d92759","Type":"ContainerStarted","Data":"d18ac5ce94b4ea9c894140ca7b6d7b153cc2da35c9ac6d18cdc165a51f3b0d26"}
Jul 28 01:09:03.463126 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:09:03.463082    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-g8zj6" podUID="f53d5a5f-7a61-468f-8fd6-09912822227d"
Jul 28 01:09:06.450857 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.450817    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4pjfd" event={"ID":"1d7ebd81-b836-4d70-926d-52fbc1e6eb80","Type":"ContainerStarted","Data":"6b77c05f54b56edba5715cfc5c1fd0283e49f2a7aab3d786447cf9c5467df803"}
Jul 28 01:09:06.451357 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.450886    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4pjfd" event={"ID":"1d7ebd81-b836-4d70-926d-52fbc1e6eb80","Type":"ContainerStarted","Data":"e8da96a7c8c69f236cacfe8db96f69f4022384d07f77a6b6c301703b9db6c300"}
Jul 28 01:09:06.451357 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.450945    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:09:06.452893 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.452865    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp" event={"ID":"4fb40ff7-f19b-4569-83fe-ffe6f49542dc","Type":"ContainerStarted","Data":"423602cc8c583840df2cb12a98a822126392b9059fa5149fc083f2730897132c"}
Jul 28 01:09:06.456174 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.456149    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv" event={"ID":"95135ee8-17bd-461f-9939-5e7cfb6bfa10","Type":"ContainerStarted","Data":"1be7771a0e847207379727523de640684a4ffaa5ccb88548478f3213d8e5e428"}
Jul 28 01:09:06.456291 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.456208    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv" event={"ID":"95135ee8-17bd-461f-9939-5e7cfb6bfa10","Type":"ContainerStarted","Data":"ea81f1d568383b13a4a04f8f20ae37c6652570eacf560aecff0de24ce29d94fa"}
Jul 28 01:09:06.456291 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.456241    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv" event={"ID":"95135ee8-17bd-461f-9939-5e7cfb6bfa10","Type":"ContainerStarted","Data":"e5a3fcc6d237ff0d9f27ec71767ae15b2d4d83bf061919c69f7591b4e07c4299"}
Jul 28 01:09:06.458633 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.458611    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2" event={"ID":"5e6c8e5c-a396-42b4-9820-101ec6d764b5","Type":"ContainerStarted","Data":"5394124b347f8c940b89b985a1ba1e9d6af351be276635a1a510aa65e298e4db"}
Jul 28 01:09:06.460351 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.460325    2565 generic.go:356] "Generic (PLEG): container finished" podID="88b2580a-9d15-45c2-92be-5ab54f563997" containerID="ba7df8401d7092b2be7029e326951a1535e629daf6de13900be1a678dbedaf50" exitCode=0
Jul 28 01:09:06.460443 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.460374    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88b2580a-9d15-45c2-92be-5ab54f563997","Type":"ContainerDied","Data":"ba7df8401d7092b2be7029e326951a1535e629daf6de13900be1a678dbedaf50"}
Jul 28 01:09:06.467715 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.466197    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4pjfd" podStartSLOduration=257.909785566 podStartE2EDuration="4m23.466182183s" totalImagesPullingTime="5.556396617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:09:06.465736314 +0000 UTC m=+297.510630296" watchObservedRunningTime="2026-07-28 01:09:06.466182183 +0000 UTC m=+297.511076152"
Jul 28 01:09:06.467715 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.466681    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-466qt" podStartSLOduration=12.130788604 podStartE2EDuration="13.466668186s" totalImagesPullingTime="1.191721649s" totalInitContainerRuntime="144.157933ms" isStatefulPod=true podCreationTimestamp="2026-07-28 01:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:09:02.455261603 +0000 UTC m=+293.500155576" watchObservedRunningTime="2026-07-28 01:09:06.466668186 +0000 UTC m=+297.511562156"
Jul 28 01:09:06.514324 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.514262    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7cccb99ffb-kmjvv" podStartSLOduration=3.083521511 podStartE2EDuration="8.514245556s" totalImagesPullingTime="5.430724045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:09:06.512604607 +0000 UTC m=+297.557498575" watchObservedRunningTime="2026-07-28 01:09:06.514245556 +0000 UTC m=+297.559139502"
Jul 28 01:09:06.549628 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.549575    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-245x2" podStartSLOduration=9.135355419 podStartE2EDuration="14.549558184s" totalImagesPullingTime="5.414202765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:09:06.530555689 +0000 UTC m=+297.575449658" watchObservedRunningTime="2026-07-28 01:09:06.549558184 +0000 UTC m=+297.594452151"
Jul 28 01:09:06.550116 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:06.550075    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp" podStartSLOduration=4.115541834 podStartE2EDuration="9.550067863s" totalImagesPullingTime="5.434526029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:09:06.548093363 +0000 UTC m=+297.592987334" watchObservedRunningTime="2026-07-28 01:09:06.550067863 +0000 UTC m=+297.594961831"
Jul 28 01:09:10.018110 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:10.018078    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:09:10.344768 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:10.018597    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:09:10.344768 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:10.018996    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:09:10.344768 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:10.019352    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:09:10.344768 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:10.024726    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:09:10.344768 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:10.024980    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:09:10.344768 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:10.029403    2565 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 01:09:11.491558 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:11.491516    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88b2580a-9d15-45c2-92be-5ab54f563997","Type":"ContainerStarted","Data":"c6eb77a2964285b4b6594efc84f5d619c12d797e51c86079ee29b32dddb34c2a"}
Jul 28 01:09:11.492034 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:11.491591    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88b2580a-9d15-45c2-92be-5ab54f563997","Type":"ContainerStarted","Data":"1454ea429fe1753e1587ebcfa0de6c7debae5ef5a627899e7291fe5dc5de20c3"}
Jul 28 01:09:13.502706 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:13.502677    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88b2580a-9d15-45c2-92be-5ab54f563997","Type":"ContainerStarted","Data":"ddcb7c361558ad70882ec6c67f01be03166e4f020c10a278029f80af19076574"}
Jul 28 01:09:13.503079 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:13.502732    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88b2580a-9d15-45c2-92be-5ab54f563997","Type":"ContainerStarted","Data":"2d65bd564a8c3c1c940c5bbdc2fad8f76e66097d8fc5c5ded8e7049aab79dd21"}
Jul 28 01:09:13.503079 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:13.502786    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88b2580a-9d15-45c2-92be-5ab54f563997","Type":"ContainerStarted","Data":"53805c31f67037031ffb241bbc930d567ad642ddb68eb2790b5c317dfbed4062"}
Jul 28 01:09:13.503079 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:13.502811    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88b2580a-9d15-45c2-92be-5ab54f563997","Type":"ContainerStarted","Data":"a135ae90bcbc2ac8019522c04a737d8b8a04c11d48cec1b991dc60d4c71612ce"}
Jul 28 01:09:13.531547 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:13.531453    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.781828242 podStartE2EDuration="14.531437386s" totalImagesPullingTime="11.723923508s" totalInitContainerRuntime="25.685636ms" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:09:13.529547858 +0000 UTC m=+304.574441825" watchObservedRunningTime="2026-07-28 01:09:13.531437386 +0000 UTC m=+304.576331353"
Jul 28 01:09:13.962160 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:13.962124    2565 patch_prober.go:28] interesting pod/image-registry-78cbc94cd5-2kg4v container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 01:09:13.962379 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:13.962178    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v" podUID="7b0e5250-f00e-47f7-aa59-a06f76c28c76" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 01:09:14.460829 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:14.460792    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:09:14.541500 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:14.541458    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:09:16.472613 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:16.472583    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4pjfd"
Jul 28 01:09:17.672587 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:17.672549    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:09:17.673048 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:17.672632    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:09:21.421603 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:21.421573    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:09:26.539741 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.539680    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8596c5cc5-nqnhg" podUID="7123f7db-1a20-4302-9b3f-94b45d2ef12e" containerName="console" containerID="cri-o://98eb18e0ffb7b39c867f90e23751fb80621dda517aef9c5ba0fb4eb8ab70225a" gracePeriod=15
Jul 28 01:09:26.804985 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.804929    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:09:26.848547 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.848513    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-service-ca\") pod \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") "
Jul 28 01:09:26.848707 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.848597    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-config\") pod \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") "
Jul 28 01:09:26.848707 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.848629    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-oauth-serving-cert\") pod \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") "
Jul 28 01:09:26.848707 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.848658    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-serving-cert\") pod \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") "
Jul 28 01:09:26.848707 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.848686    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8f4h\" (UniqueName: \"kubernetes.io/projected/7123f7db-1a20-4302-9b3f-94b45d2ef12e-kube-api-access-r8f4h\") pod \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") "
Jul 28 01:09:26.848900 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.848727    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-oauth-config\") pod \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\" (UID: \"7123f7db-1a20-4302-9b3f-94b45d2ef12e\") "
Jul 28 01:09:26.849055 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.848928    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-service-ca" (OuterVolumeSpecName: "service-ca") pod "7123f7db-1a20-4302-9b3f-94b45d2ef12e" (UID: "7123f7db-1a20-4302-9b3f-94b45d2ef12e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:26.849171 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.849153    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-config" (OuterVolumeSpecName: "console-config") pod "7123f7db-1a20-4302-9b3f-94b45d2ef12e" (UID: "7123f7db-1a20-4302-9b3f-94b45d2ef12e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:26.849286 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.849262    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7123f7db-1a20-4302-9b3f-94b45d2ef12e" (UID: "7123f7db-1a20-4302-9b3f-94b45d2ef12e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:26.851117 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.851091    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7123f7db-1a20-4302-9b3f-94b45d2ef12e" (UID: "7123f7db-1a20-4302-9b3f-94b45d2ef12e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:09:26.851215 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.851125    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7123f7db-1a20-4302-9b3f-94b45d2ef12e" (UID: "7123f7db-1a20-4302-9b3f-94b45d2ef12e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:09:26.851215 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.851143    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7123f7db-1a20-4302-9b3f-94b45d2ef12e-kube-api-access-r8f4h" (OuterVolumeSpecName: "kube-api-access-r8f4h") pod "7123f7db-1a20-4302-9b3f-94b45d2ef12e" (UID: "7123f7db-1a20-4302-9b3f-94b45d2ef12e"). InnerVolumeSpecName "kube-api-access-r8f4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:09:26.949453 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.949424    2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:09:26.949453 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.949447    2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-config\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:09:26.949453 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.949456    2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7123f7db-1a20-4302-9b3f-94b45d2ef12e-oauth-serving-cert\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:09:26.949663 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.949467    2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-serving-cert\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:09:26.949663 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.949477    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8f4h\" (UniqueName: \"kubernetes.io/projected/7123f7db-1a20-4302-9b3f-94b45d2ef12e-kube-api-access-r8f4h\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:09:26.949663 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:26.949506    2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7123f7db-1a20-4302-9b3f-94b45d2ef12e-console-oauth-config\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:09:27.556693 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:27.556657    2565 generic.go:356] "Generic (PLEG): container finished" podID="7123f7db-1a20-4302-9b3f-94b45d2ef12e" containerID="98eb18e0ffb7b39c867f90e23751fb80621dda517aef9c5ba0fb4eb8ab70225a" exitCode=0
Jul 28 01:09:27.557230 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:27.556701    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8596c5cc5-nqnhg"
Jul 28 01:09:27.557230 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:27.556712    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8596c5cc5-nqnhg" event={"ID":"7123f7db-1a20-4302-9b3f-94b45d2ef12e","Type":"ContainerDied","Data":"98eb18e0ffb7b39c867f90e23751fb80621dda517aef9c5ba0fb4eb8ab70225a"}
Jul 28 01:09:27.557230 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:27.556760    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8596c5cc5-nqnhg" event={"ID":"7123f7db-1a20-4302-9b3f-94b45d2ef12e","Type":"ContainerDied","Data":"2ba5fc7d4cd2d8f5bbf26269ad12ab1fa34fafdfb8b7948eddbeb133380c0b8c"}
Jul 28 01:09:27.557230 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:27.556787    2565 scope.go:117] "RemoveContainer" containerID="98eb18e0ffb7b39c867f90e23751fb80621dda517aef9c5ba0fb4eb8ab70225a"
Jul 28 01:09:27.565229 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:27.565211    2565 scope.go:117] "RemoveContainer" containerID="98eb18e0ffb7b39c867f90e23751fb80621dda517aef9c5ba0fb4eb8ab70225a"
Jul 28 01:09:27.565472 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:09:27.565454    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98eb18e0ffb7b39c867f90e23751fb80621dda517aef9c5ba0fb4eb8ab70225a\": container with ID starting with 98eb18e0ffb7b39c867f90e23751fb80621dda517aef9c5ba0fb4eb8ab70225a not found: ID does not exist" containerID="98eb18e0ffb7b39c867f90e23751fb80621dda517aef9c5ba0fb4eb8ab70225a"
Jul 28 01:09:27.565554 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:27.565491    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98eb18e0ffb7b39c867f90e23751fb80621dda517aef9c5ba0fb4eb8ab70225a"} err="failed to get container status \"98eb18e0ffb7b39c867f90e23751fb80621dda517aef9c5ba0fb4eb8ab70225a\": rpc error: code = NotFound desc = could not find container \"98eb18e0ffb7b39c867f90e23751fb80621dda517aef9c5ba0fb4eb8ab70225a\": container with ID starting with 98eb18e0ffb7b39c867f90e23751fb80621dda517aef9c5ba0fb4eb8ab70225a not found: ID does not exist"
Jul 28 01:09:27.572908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:27.572889    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8596c5cc5-nqnhg"]
Jul 28 01:09:27.576386 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:27.576365    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8596c5cc5-nqnhg"]
Jul 28 01:09:29.470401 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:29.470368    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7123f7db-1a20-4302-9b3f-94b45d2ef12e" path="/var/lib/kubelet/pods/7123f7db-1a20-4302-9b3f-94b45d2ef12e/volumes"
Jul 28 01:09:30.181713 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:30.181676    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78cbc94cd5-2kg4v"]
Jul 28 01:09:35.533381 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:35.533326    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nbqk2_2be7258c-f1ce-42df-8769-90258eda6a2b/serve-healthcheck-canary/0.log"
Jul 28 01:09:37.677665 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:37.677636    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:09:37.681572 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:37.681552    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-79755cbbf5-f99cp"
Jul 28 01:09:39.603339 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:39.603305    2565 generic.go:356] "Generic (PLEG): container finished" podID="70870e41-edec-4901-8c63-9fab7dee0173" containerID="55a2315cafb57b9e0c15e2e2ccf0af318d0c73ffb462690d4a9843fb7d83e131" exitCode=0
Jul 28 01:09:39.603738 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:39.603375    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4" event={"ID":"70870e41-edec-4901-8c63-9fab7dee0173","Type":"ContainerDied","Data":"55a2315cafb57b9e0c15e2e2ccf0af318d0c73ffb462690d4a9843fb7d83e131"}
Jul 28 01:09:39.603738 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:39.603695    2565 scope.go:117] "RemoveContainer" containerID="55a2315cafb57b9e0c15e2e2ccf0af318d0c73ffb462690d4a9843fb7d83e131"
Jul 28 01:09:40.609150 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:40.609110    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-zpfq4" event={"ID":"70870e41-edec-4901-8c63-9fab7dee0173","Type":"ContainerStarted","Data":"74653b9f547570882df8d0b7fcbc8f08201508008dcd21bad3c98c41a7ecca42"}
Jul 28 01:09:55.207584 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:55.207540    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v" podUID="7b0e5250-f00e-47f7-aa59-a06f76c28c76" containerName="registry" containerID="cri-o://1f9540ebc012b11906daca24f29dd26647712062344bf9314da0ee55f5ec69c0" gracePeriod=30
Jul 28 01:09:56.451109 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.451083    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:09:56.613201 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.613161    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7b0e5250-f00e-47f7-aa59-a06f76c28c76-ca-trust-extracted\") pod \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") "
Jul 28 01:09:56.613363 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.613213    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls\") pod \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") "
Jul 28 01:09:56.613363 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.613237    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-bound-sa-token\") pod \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") "
Jul 28 01:09:56.613363 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.613263    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7b0e5250-f00e-47f7-aa59-a06f76c28c76-installation-pull-secrets\") pod \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") "
Jul 28 01:09:56.613363 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.613300    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l48bg\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-kube-api-access-l48bg\") pod \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") "
Jul 28 01:09:56.613363 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.613328    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7b0e5250-f00e-47f7-aa59-a06f76c28c76-trusted-ca\") pod \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") "
Jul 28 01:09:56.613639 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.613384    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-certificates\") pod \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") "
Jul 28 01:09:56.613639 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.613416    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7b0e5250-f00e-47f7-aa59-a06f76c28c76-image-registry-private-configuration\") pod \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\" (UID: \"7b0e5250-f00e-47f7-aa59-a06f76c28c76\") "
Jul 28 01:09:56.614255 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.614217    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b0e5250-f00e-47f7-aa59-a06f76c28c76-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7b0e5250-f00e-47f7-aa59-a06f76c28c76" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:56.614502 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.614453    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7b0e5250-f00e-47f7-aa59-a06f76c28c76" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:56.616023 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.615981    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b0e5250-f00e-47f7-aa59-a06f76c28c76-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7b0e5250-f00e-47f7-aa59-a06f76c28c76" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:09:56.616023 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.615993    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7b0e5250-f00e-47f7-aa59-a06f76c28c76" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:09:56.616023 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.616014    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7b0e5250-f00e-47f7-aa59-a06f76c28c76" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:09:56.616245 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.616225    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-kube-api-access-l48bg" (OuterVolumeSpecName: "kube-api-access-l48bg") pod "7b0e5250-f00e-47f7-aa59-a06f76c28c76" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76"). InnerVolumeSpecName "kube-api-access-l48bg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:09:56.616327 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.616303    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b0e5250-f00e-47f7-aa59-a06f76c28c76-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7b0e5250-f00e-47f7-aa59-a06f76c28c76" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:09:56.622264 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.622237    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b0e5250-f00e-47f7-aa59-a06f76c28c76-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7b0e5250-f00e-47f7-aa59-a06f76c28c76" (UID: "7b0e5250-f00e-47f7-aa59-a06f76c28c76"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:09:56.675501 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.675456    2565 generic.go:356] "Generic (PLEG): container finished" podID="7b0e5250-f00e-47f7-aa59-a06f76c28c76" containerID="1f9540ebc012b11906daca24f29dd26647712062344bf9314da0ee55f5ec69c0" exitCode=0
Jul 28 01:09:56.675595 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.675524    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v"
Jul 28 01:09:56.675595 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.675532    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v" event={"ID":"7b0e5250-f00e-47f7-aa59-a06f76c28c76","Type":"ContainerDied","Data":"1f9540ebc012b11906daca24f29dd26647712062344bf9314da0ee55f5ec69c0"}
Jul 28 01:09:56.675595 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.675576    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78cbc94cd5-2kg4v" event={"ID":"7b0e5250-f00e-47f7-aa59-a06f76c28c76","Type":"ContainerDied","Data":"2fc30d29dcf6f036fbe9ad53d99e07387931161f42a073bd36d7615c3a165d14"}
Jul 28 01:09:56.675595 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.675595    2565 scope.go:117] "RemoveContainer" containerID="1f9540ebc012b11906daca24f29dd26647712062344bf9314da0ee55f5ec69c0"
Jul 28 01:09:56.684164 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.684151    2565 scope.go:117] "RemoveContainer" containerID="1f9540ebc012b11906daca24f29dd26647712062344bf9314da0ee55f5ec69c0"
Jul 28 01:09:56.684416 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:09:56.684398    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f9540ebc012b11906daca24f29dd26647712062344bf9314da0ee55f5ec69c0\": container with ID starting with 1f9540ebc012b11906daca24f29dd26647712062344bf9314da0ee55f5ec69c0 not found: ID does not exist" containerID="1f9540ebc012b11906daca24f29dd26647712062344bf9314da0ee55f5ec69c0"
Jul 28 01:09:56.684501 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.684425    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f9540ebc012b11906daca24f29dd26647712062344bf9314da0ee55f5ec69c0"} err="failed to get container status \"1f9540ebc012b11906daca24f29dd26647712062344bf9314da0ee55f5ec69c0\": rpc error: code = NotFound desc = could not find container \"1f9540ebc012b11906daca24f29dd26647712062344bf9314da0ee55f5ec69c0\": container with ID starting with 1f9540ebc012b11906daca24f29dd26647712062344bf9314da0ee55f5ec69c0 not found: ID does not exist"
Jul 28 01:09:56.696914 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.696882    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78cbc94cd5-2kg4v"]
Jul 28 01:09:56.700627 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.700608    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78cbc94cd5-2kg4v"]
Jul 28 01:09:56.714385 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.714360    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l48bg\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-kube-api-access-l48bg\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:09:56.714385 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.714383    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7b0e5250-f00e-47f7-aa59-a06f76c28c76-trusted-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:09:56.714536 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.714399    2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-certificates\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:09:56.714536 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.714411    2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7b0e5250-f00e-47f7-aa59-a06f76c28c76-image-registry-private-configuration\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:09:56.714536 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.714425    2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7b0e5250-f00e-47f7-aa59-a06f76c28c76-ca-trust-extracted\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:09:56.714536 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.714438    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-registry-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:09:56.714536 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.714450    2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7b0e5250-f00e-47f7-aa59-a06f76c28c76-bound-sa-token\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:09:56.714536 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:56.714461    2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7b0e5250-f00e-47f7-aa59-a06f76c28c76-installation-pull-secrets\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:09:57.466366 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:57.466329    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b0e5250-f00e-47f7-aa59-a06f76c28c76" path="/var/lib/kubelet/pods/7b0e5250-f00e-47f7-aa59-a06f76c28c76/volumes"
Jul 28 01:09:59.541602 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:59.541521    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:09:59.562457 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:59.562436    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:09:59.702590 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:09:59.702565    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 01:10:23.246767 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:10:23.246731    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:10:23.249137 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:10:23.249111    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f53d5a5f-7a61-468f-8fd6-09912822227d-metrics-certs\") pod \"network-metrics-daemon-g8zj6\" (UID: \"f53d5a5f-7a61-468f-8fd6-09912822227d\") " pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:10:23.464672 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:10:23.464637    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qvhk6\""
Jul 28 01:10:23.472874 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:10:23.472855    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g8zj6"
Jul 28 01:10:23.596583 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:10:23.596547    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g8zj6"]
Jul 28 01:10:23.598666 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:10:23.598631    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf53d5a5f_7a61_468f_8fd6_09912822227d.slice/crio-eb7a956819fe9600a1a42174673f79d5b17b04e07d1b34720f8412617e98940e WatchSource:0}: Error finding container eb7a956819fe9600a1a42174673f79d5b17b04e07d1b34720f8412617e98940e: Status 404 returned error can't find the container with id eb7a956819fe9600a1a42174673f79d5b17b04e07d1b34720f8412617e98940e
Jul 28 01:10:23.600508 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:10:23.600470    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:10:23.782998 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:10:23.782920    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g8zj6" event={"ID":"f53d5a5f-7a61-468f-8fd6-09912822227d","Type":"ContainerStarted","Data":"eb7a956819fe9600a1a42174673f79d5b17b04e07d1b34720f8412617e98940e"}
Jul 28 01:10:24.788892 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:10:24.788801    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g8zj6" event={"ID":"f53d5a5f-7a61-468f-8fd6-09912822227d","Type":"ContainerStarted","Data":"c506adc93e3154a589225eaf02b754accea5fb3b158f3a5f0d0ce8e36299b51d"}
Jul 28 01:10:24.788892 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:10:24.788857    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g8zj6" event={"ID":"f53d5a5f-7a61-468f-8fd6-09912822227d","Type":"ContainerStarted","Data":"b125f7c6dcfb24773229eaf01e3051e221e61dc0b9d6756b019100935c7454da"}
Jul 28 01:10:24.805340 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:10:24.805293    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-g8zj6" podStartSLOduration=374.975539965 podStartE2EDuration="6m15.805278825s" totalImagesPullingTime="829.73886ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:10:24.804279369 +0000 UTC m=+375.849173337" watchObservedRunningTime="2026-07-28 01:10:24.805278825 +0000 UTC m=+375.850172792"
Jul 28 01:11:35.264334 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.264258    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mvljm"]
Jul 28 01:11:35.264785 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.264571    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b0e5250-f00e-47f7-aa59-a06f76c28c76" containerName="registry"
Jul 28 01:11:35.264785 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.264582    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b0e5250-f00e-47f7-aa59-a06f76c28c76" containerName="registry"
Jul 28 01:11:35.264785 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.264600    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7123f7db-1a20-4302-9b3f-94b45d2ef12e" containerName="console"
Jul 28 01:11:35.264785 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.264605    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7123f7db-1a20-4302-9b3f-94b45d2ef12e" containerName="console"
Jul 28 01:11:35.264785 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.264651    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b0e5250-f00e-47f7-aa59-a06f76c28c76" containerName="registry"
Jul 28 01:11:35.264785 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.264661    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7123f7db-1a20-4302-9b3f-94b45d2ef12e" containerName="console"
Jul 28 01:11:35.266515 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.266499    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mvljm"
Jul 28 01:11:35.269813 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.269789    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 01:11:35.271080 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.271061    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 01:11:35.271141 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.271061    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bxbbz\""
Jul 28 01:11:35.286100 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.286071    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mvljm"]
Jul 28 01:11:35.337000 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.336971    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2qwb\" (UniqueName: \"kubernetes.io/projected/0291b627-3d81-4b7e-a7f9-f5138d8ca430-kube-api-access-p2qwb\") pod \"cert-manager-webhook-6486ffd997-mvljm\" (UID: \"0291b627-3d81-4b7e-a7f9-f5138d8ca430\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mvljm"
Jul 28 01:11:35.337114 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.337006    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0291b627-3d81-4b7e-a7f9-f5138d8ca430-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mvljm\" (UID: \"0291b627-3d81-4b7e-a7f9-f5138d8ca430\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mvljm"
Jul 28 01:11:35.438042 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.438013    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2qwb\" (UniqueName: \"kubernetes.io/projected/0291b627-3d81-4b7e-a7f9-f5138d8ca430-kube-api-access-p2qwb\") pod \"cert-manager-webhook-6486ffd997-mvljm\" (UID: \"0291b627-3d81-4b7e-a7f9-f5138d8ca430\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mvljm"
Jul 28 01:11:35.438162 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.438048    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0291b627-3d81-4b7e-a7f9-f5138d8ca430-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mvljm\" (UID: \"0291b627-3d81-4b7e-a7f9-f5138d8ca430\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mvljm"
Jul 28 01:11:35.446081 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.446052    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0291b627-3d81-4b7e-a7f9-f5138d8ca430-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mvljm\" (UID: \"0291b627-3d81-4b7e-a7f9-f5138d8ca430\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mvljm"
Jul 28 01:11:35.446218 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.446197    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2qwb\" (UniqueName: \"kubernetes.io/projected/0291b627-3d81-4b7e-a7f9-f5138d8ca430-kube-api-access-p2qwb\") pod \"cert-manager-webhook-6486ffd997-mvljm\" (UID: \"0291b627-3d81-4b7e-a7f9-f5138d8ca430\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mvljm"
Jul 28 01:11:35.575678 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.575644    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mvljm"
Jul 28 01:11:35.701236 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:35.701210    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mvljm"]
Jul 28 01:11:35.703183 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:11:35.703155    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0291b627_3d81_4b7e_a7f9_f5138d8ca430.slice/crio-9bc060d6fd8547589215df70dd91620b58505155882e076cd9615f25466f263c WatchSource:0}: Error finding container 9bc060d6fd8547589215df70dd91620b58505155882e076cd9615f25466f263c: Status 404 returned error can't find the container with id 9bc060d6fd8547589215df70dd91620b58505155882e076cd9615f25466f263c
Jul 28 01:11:36.059365 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:36.059335    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mvljm" event={"ID":"0291b627-3d81-4b7e-a7f9-f5138d8ca430","Type":"ContainerStarted","Data":"9bc060d6fd8547589215df70dd91620b58505155882e076cd9615f25466f263c"}
Jul 28 01:11:49.112656 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:49.112623    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mvljm" event={"ID":"0291b627-3d81-4b7e-a7f9-f5138d8ca430","Type":"ContainerStarted","Data":"196f4c54b04f93e7d6cee1e06392ee87e5e855d5886749070db242a41f0d6f61"}
Jul 28 01:11:49.113046 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:49.112691    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mvljm"
Jul 28 01:11:49.131148 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:49.131097    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-mvljm" podStartSLOduration=1.358764976 podStartE2EDuration="14.131082845s" totalImagesPullingTime="12.772317869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:11:49.128738121 +0000 UTC m=+460.173632088" watchObservedRunningTime="2026-07-28 01:11:49.131082845 +0000 UTC m=+460.175976813"
Jul 28 01:11:55.119539 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:11:55.119500    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mvljm"
Jul 28 01:12:12.332253 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.332219    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"]
Jul 28 01:12:12.334436 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.334420    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:12.338421 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.338395    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 01:12:12.338573 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.338418    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 01:12:12.338573 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.338444    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:12:12.338573 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.338409    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-v7bh5\""
Jul 28 01:12:12.338573 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.338424    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 01:12:12.338573 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.338527    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 01:12:12.343377 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.343354    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"]
Jul 28 01:12:12.448298 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.448260    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d144fcce-f5ef-4688-b7d5-70a518a27b7a-manager-config\") pod \"lws-controller-manager-6d99f9899c-ndlk2\" (UID: \"d144fcce-f5ef-4688-b7d5-70a518a27b7a\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:12.448477 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.448329    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d144fcce-f5ef-4688-b7d5-70a518a27b7a-cert\") pod \"lws-controller-manager-6d99f9899c-ndlk2\" (UID: \"d144fcce-f5ef-4688-b7d5-70a518a27b7a\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:12.448477 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.448391    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gwxb\" (UniqueName: \"kubernetes.io/projected/d144fcce-f5ef-4688-b7d5-70a518a27b7a-kube-api-access-9gwxb\") pod \"lws-controller-manager-6d99f9899c-ndlk2\" (UID: \"d144fcce-f5ef-4688-b7d5-70a518a27b7a\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:12.448477 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.448462    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d144fcce-f5ef-4688-b7d5-70a518a27b7a-metrics-cert\") pod \"lws-controller-manager-6d99f9899c-ndlk2\" (UID: \"d144fcce-f5ef-4688-b7d5-70a518a27b7a\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:12.549531 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.549472    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d144fcce-f5ef-4688-b7d5-70a518a27b7a-manager-config\") pod \"lws-controller-manager-6d99f9899c-ndlk2\" (UID: \"d144fcce-f5ef-4688-b7d5-70a518a27b7a\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:12.549674 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.549553    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d144fcce-f5ef-4688-b7d5-70a518a27b7a-cert\") pod \"lws-controller-manager-6d99f9899c-ndlk2\" (UID: \"d144fcce-f5ef-4688-b7d5-70a518a27b7a\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:12.549674 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.549595    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gwxb\" (UniqueName: \"kubernetes.io/projected/d144fcce-f5ef-4688-b7d5-70a518a27b7a-kube-api-access-9gwxb\") pod \"lws-controller-manager-6d99f9899c-ndlk2\" (UID: \"d144fcce-f5ef-4688-b7d5-70a518a27b7a\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:12.549674 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.549617    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d144fcce-f5ef-4688-b7d5-70a518a27b7a-metrics-cert\") pod \"lws-controller-manager-6d99f9899c-ndlk2\" (UID: \"d144fcce-f5ef-4688-b7d5-70a518a27b7a\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:12.550208 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.550184    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d144fcce-f5ef-4688-b7d5-70a518a27b7a-manager-config\") pod \"lws-controller-manager-6d99f9899c-ndlk2\" (UID: \"d144fcce-f5ef-4688-b7d5-70a518a27b7a\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:12.552114 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.552089    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d144fcce-f5ef-4688-b7d5-70a518a27b7a-cert\") pod \"lws-controller-manager-6d99f9899c-ndlk2\" (UID: \"d144fcce-f5ef-4688-b7d5-70a518a27b7a\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:12.552198 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.552137    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d144fcce-f5ef-4688-b7d5-70a518a27b7a-metrics-cert\") pod \"lws-controller-manager-6d99f9899c-ndlk2\" (UID: \"d144fcce-f5ef-4688-b7d5-70a518a27b7a\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:12.558738 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.558707    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gwxb\" (UniqueName: \"kubernetes.io/projected/d144fcce-f5ef-4688-b7d5-70a518a27b7a-kube-api-access-9gwxb\") pod \"lws-controller-manager-6d99f9899c-ndlk2\" (UID: \"d144fcce-f5ef-4688-b7d5-70a518a27b7a\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:12.644349 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.644277    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:12.772845 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:12.772818    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"]
Jul 28 01:12:12.775116 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:12:12.775086    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd144fcce_f5ef_4688_b7d5_70a518a27b7a.slice/crio-78a89b98f767d16c4d757376da1f0cd98fc1db0be72a7567c553aa4b590b54b5 WatchSource:0}: Error finding container 78a89b98f767d16c4d757376da1f0cd98fc1db0be72a7567c553aa4b590b54b5: Status 404 returned error can't find the container with id 78a89b98f767d16c4d757376da1f0cd98fc1db0be72a7567c553aa4b590b54b5
Jul 28 01:12:13.206003 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:13.205972    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2" event={"ID":"d144fcce-f5ef-4688-b7d5-70a518a27b7a","Type":"ContainerStarted","Data":"78a89b98f767d16c4d757376da1f0cd98fc1db0be72a7567c553aa4b590b54b5"}
Jul 28 01:12:16.224753 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.224714    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2" event={"ID":"d144fcce-f5ef-4688-b7d5-70a518a27b7a","Type":"ContainerStarted","Data":"9fddecae6dbb00c5f471c8d84f2c7cf32bd080bd7648306d91226ccb9d89f6b9"}
Jul 28 01:12:16.225207 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.224857    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:16.247568 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.247510    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2" podStartSLOduration=1.866660435 podStartE2EDuration="4.247494728s" totalImagesPullingTime="2.380834293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:12:16.246475537 +0000 UTC m=+487.291369506" watchObservedRunningTime="2026-07-28 01:12:16.247494728 +0000 UTC m=+487.292388687"
Jul 28 01:12:16.464014 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.463972    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"]
Jul 28 01:12:16.467237 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.467215    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.469926 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.469902    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 01:12:16.470030 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.469953    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-b7f4g\""
Jul 28 01:12:16.470030 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.470009    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 01:12:16.470188 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.470171    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 01:12:16.470370 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.470355    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 01:12:16.486086 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.486035    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"]
Jul 28 01:12:16.584714 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.584686    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.584827 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.584716    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-tmp\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.584827 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.584739    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-manifests\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.584910 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.584867    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.584963 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.584929    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msd5w\" (UniqueName: \"kubernetes.io/projected/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-kube-api-access-msd5w\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.685322 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.685302    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.685445 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.685335    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-tmp\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.685445 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.685368    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-manifests\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.685445 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.685417    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.685637 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.685557    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msd5w\" (UniqueName: \"kubernetes.io/projected/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-kube-api-access-msd5w\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.685827 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.685800    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-manifests\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.687860 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.687842    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-tmp\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.687941 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.687932    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.688010 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.687969    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.693647 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.693618    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msd5w\" (UniqueName: \"kubernetes.io/projected/f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4-kube-api-access-msd5w\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk\" (UID: \"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.777673 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.777609    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:16.907528 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:16.907458    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"]
Jul 28 01:12:16.909653 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:12:16.909625    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2fae8da_c7f3_44b9_9cee_a3cfaa975ef4.slice/crio-5f9a3d4421367b75448cb3847fd06a4dcd52b6f0a9ebf04ff00ef4cc43043b4d WatchSource:0}: Error finding container 5f9a3d4421367b75448cb3847fd06a4dcd52b6f0a9ebf04ff00ef4cc43043b4d: Status 404 returned error can't find the container with id 5f9a3d4421367b75448cb3847fd06a4dcd52b6f0a9ebf04ff00ef4cc43043b4d
Jul 28 01:12:17.230378 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:17.230338    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk" event={"ID":"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4","Type":"ContainerStarted","Data":"5f9a3d4421367b75448cb3847fd06a4dcd52b6f0a9ebf04ff00ef4cc43043b4d"}
Jul 28 01:12:20.245802 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:20.245718    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk" event={"ID":"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4","Type":"ContainerStarted","Data":"4ad40845776c2fbd67dbd7150d16491d4f57653b3ab3f46146e747e6fc51b434"}
Jul 28 01:12:21.251462 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:21.251375    2565 generic.go:356] "Generic (PLEG): container finished" podID="f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4" containerID="4ad40845776c2fbd67dbd7150d16491d4f57653b3ab3f46146e747e6fc51b434" exitCode=0
Jul 28 01:12:21.251462 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:21.251449    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk" event={"ID":"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4","Type":"ContainerDied","Data":"4ad40845776c2fbd67dbd7150d16491d4f57653b3ab3f46146e747e6fc51b434"}
Jul 28 01:12:22.257215 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:22.257183    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk" event={"ID":"f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4","Type":"ContainerStarted","Data":"7010d18b71e2d261eff71c2c1f28dd2094d44beda725d691d0ceb6470a78773a"}
Jul 28 01:12:22.257596 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:22.257319    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:22.279036 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:22.278965    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk" podStartSLOduration=1.9464091940000001 podStartE2EDuration="6.278945805s" totalImagesPullingTime="3.331332391s" totalInitContainerRuntime="1.00120422s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:12:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:12:22.277536663 +0000 UTC m=+493.322430631" watchObservedRunningTime="2026-07-28 01:12:22.278945805 +0000 UTC m=+493.323839773"
Jul 28 01:12:27.233086 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:27.233053    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-ndlk2"
Jul 28 01:12:33.263779 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:33.263745    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk"
Jul 28 01:12:46.361303 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.361275    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs"]
Jul 28 01:12:46.364969 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.364949    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs"
Jul 28 01:12:46.367715 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.367692    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-lspb9\""
Jul 28 01:12:46.367715 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.367704    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 01:12:46.368926 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.368904    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 01:12:46.368926 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.368915    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 01:12:46.369082 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.368958    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 01:12:46.374799 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.374779    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs"]
Jul 28 01:12:46.441900 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.441868    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwvlg\" (UniqueName: \"kubernetes.io/projected/ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68-kube-api-access-kwvlg\") pod \"kube-auth-proxy-5f6ffdb965-shxvs\" (UID: \"ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68\") " pod="openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs"
Jul 28 01:12:46.442072 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.441914    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68-tls-certs\") pod \"kube-auth-proxy-5f6ffdb965-shxvs\" (UID: \"ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68\") " pod="openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs"
Jul 28 01:12:46.442072 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.441929    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68-tmp\") pod \"kube-auth-proxy-5f6ffdb965-shxvs\" (UID: \"ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68\") " pod="openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs"
Jul 28 01:12:46.543190 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.543157    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwvlg\" (UniqueName: \"kubernetes.io/projected/ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68-kube-api-access-kwvlg\") pod \"kube-auth-proxy-5f6ffdb965-shxvs\" (UID: \"ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68\") " pod="openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs"
Jul 28 01:12:46.543359 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.543219    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68-tls-certs\") pod \"kube-auth-proxy-5f6ffdb965-shxvs\" (UID: \"ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68\") " pod="openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs"
Jul 28 01:12:46.543359 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.543243    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68-tmp\") pod \"kube-auth-proxy-5f6ffdb965-shxvs\" (UID: \"ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68\") " pod="openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs"
Jul 28 01:12:46.545587 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.545563    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68-tmp\") pod \"kube-auth-proxy-5f6ffdb965-shxvs\" (UID: \"ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68\") " pod="openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs"
Jul 28 01:12:46.545797 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.545780    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68-tls-certs\") pod \"kube-auth-proxy-5f6ffdb965-shxvs\" (UID: \"ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68\") " pod="openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs"
Jul 28 01:12:46.559237 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.559210    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwvlg\" (UniqueName: \"kubernetes.io/projected/ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68-kube-api-access-kwvlg\") pod \"kube-auth-proxy-5f6ffdb965-shxvs\" (UID: \"ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68\") " pod="openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs"
Jul 28 01:12:46.676039 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.675968    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs"
Jul 28 01:12:46.822362 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:46.822338    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs"]
Jul 28 01:12:46.825766 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:12:46.825730    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea9c1c2d_1ed4_4fe5_a2c0_eaff0403ae68.slice/crio-7b6527fbb429329754520dc5af87ff2f0ae8cbbaf44b692a98528fbec6680b12 WatchSource:0}: Error finding container 7b6527fbb429329754520dc5af87ff2f0ae8cbbaf44b692a98528fbec6680b12: Status 404 returned error can't find the container with id 7b6527fbb429329754520dc5af87ff2f0ae8cbbaf44b692a98528fbec6680b12
Jul 28 01:12:47.366561 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:47.366453    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs" event={"ID":"ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68","Type":"ContainerStarted","Data":"7b6527fbb429329754520dc5af87ff2f0ae8cbbaf44b692a98528fbec6680b12"}
Jul 28 01:12:50.381825 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:50.381789    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs" event={"ID":"ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68","Type":"ContainerStarted","Data":"1af5e2fbeae5176900db6964adbb34524f0748705af42b0f501249b12318afdf"}
Jul 28 01:12:50.397653 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:12:50.397608    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5f6ffdb965-shxvs" podStartSLOduration=1.542704621 podStartE2EDuration="4.397595557s" totalImagesPullingTime="2.854890936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:12:50.396507706 +0000 UTC m=+521.441401671" watchObservedRunningTime="2026-07-28 01:12:50.397595557 +0000 UTC m=+521.442489524"
Jul 28 01:14:10.046295 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:10.046182    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:14:10.049713 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:10.047240    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:14:10.049713 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:10.047539    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:14:10.049713 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:10.048544    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:14:10.053229 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:10.053207    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:14:10.054493 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:10.054444    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:14:17.175650 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.175577    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b9d549b4-lqlbb"]
Jul 28 01:14:17.178705 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.178690    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.181898 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.181874    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 01:14:17.182011 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.181967    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 01:14:17.183600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.183580    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 01:14:17.183732 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.183704    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dhb6l\""
Jul 28 01:14:17.183812 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.183785    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 01:14:17.183868 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.183824    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 01:14:17.196058 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.196036    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 01:14:17.197783 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.197761    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b9d549b4-lqlbb"]
Jul 28 01:14:17.306578 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.306543    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2228e675-8282-41d9-8905-c5505b17318a-console-oauth-config\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.306732 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.306606    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2228e675-8282-41d9-8905-c5505b17318a-oauth-serving-cert\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.306732 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.306667    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2228e675-8282-41d9-8905-c5505b17318a-console-config\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.306732 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.306686    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2228e675-8282-41d9-8905-c5505b17318a-console-serving-cert\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.306732 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.306708    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2228e675-8282-41d9-8905-c5505b17318a-service-ca\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.306865 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.306745    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2228e675-8282-41d9-8905-c5505b17318a-trusted-ca-bundle\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.306865 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.306778    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4grxf\" (UniqueName: \"kubernetes.io/projected/2228e675-8282-41d9-8905-c5505b17318a-kube-api-access-4grxf\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.407790 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.407755    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2228e675-8282-41d9-8905-c5505b17318a-oauth-serving-cert\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.407947 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.407801    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2228e675-8282-41d9-8905-c5505b17318a-console-config\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.407947 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.407827    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2228e675-8282-41d9-8905-c5505b17318a-console-serving-cert\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.407947 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.407847    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2228e675-8282-41d9-8905-c5505b17318a-service-ca\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.407947 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.407885    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2228e675-8282-41d9-8905-c5505b17318a-trusted-ca-bundle\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.407947 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.407917    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4grxf\" (UniqueName: \"kubernetes.io/projected/2228e675-8282-41d9-8905-c5505b17318a-kube-api-access-4grxf\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.408194 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.407951    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2228e675-8282-41d9-8905-c5505b17318a-console-oauth-config\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.408727 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.408696    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2228e675-8282-41d9-8905-c5505b17318a-console-config\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.408854 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.408793    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2228e675-8282-41d9-8905-c5505b17318a-oauth-serving-cert\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.408967 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.408947    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2228e675-8282-41d9-8905-c5505b17318a-trusted-ca-bundle\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.409028 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.408952    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2228e675-8282-41d9-8905-c5505b17318a-service-ca\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.410525 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.410502    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2228e675-8282-41d9-8905-c5505b17318a-console-serving-cert\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.410604 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.410571    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2228e675-8282-41d9-8905-c5505b17318a-console-oauth-config\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.415783 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.415764    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4grxf\" (UniqueName: \"kubernetes.io/projected/2228e675-8282-41d9-8905-c5505b17318a-kube-api-access-4grxf\") pod \"console-7b9d549b4-lqlbb\" (UID: \"2228e675-8282-41d9-8905-c5505b17318a\") " pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.488855 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.488781    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:17.615509 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.615467    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b9d549b4-lqlbb"]
Jul 28 01:14:17.618330 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:14:17.618295    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2228e675_8282_41d9_8905_c5505b17318a.slice/crio-470e2d78a0267f3b15da66debafba8699dc0ad7753a5e4b06789cbe7cf4803ad WatchSource:0}: Error finding container 470e2d78a0267f3b15da66debafba8699dc0ad7753a5e4b06789cbe7cf4803ad: Status 404 returned error can't find the container with id 470e2d78a0267f3b15da66debafba8699dc0ad7753a5e4b06789cbe7cf4803ad
Jul 28 01:14:17.745034 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.744958    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b9d549b4-lqlbb" event={"ID":"2228e675-8282-41d9-8905-c5505b17318a","Type":"ContainerStarted","Data":"1af84583ecfe69d1b3ac4ced8496623ff27e7bbb45f3ef45ec54c81cfb6357a2"}
Jul 28 01:14:17.745034 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.745013    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b9d549b4-lqlbb" event={"ID":"2228e675-8282-41d9-8905-c5505b17318a","Type":"ContainerStarted","Data":"470e2d78a0267f3b15da66debafba8699dc0ad7753a5e4b06789cbe7cf4803ad"}
Jul 28 01:14:17.762739 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:17.762686    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b9d549b4-lqlbb" podStartSLOduration=0.762670819 podStartE2EDuration="762.670819ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:14:17.761606925 +0000 UTC m=+608.806500896" watchObservedRunningTime="2026-07-28 01:14:17.762670819 +0000 UTC m=+608.807564836"
Jul 28 01:14:27.489907 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:27.489878    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:27.490347 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:27.489931    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:27.494751 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:27.494732    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:27.795578 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:27.795475    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b9d549b4-lqlbb"
Jul 28 01:14:44.269672 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.269587    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"]
Jul 28 01:14:44.272811 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.272790    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"
Jul 28 01:14:44.277401 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.277379    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 28 01:14:44.277559 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.277542    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 01:14:44.277786 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.277768    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 28 01:14:44.278720 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.278698    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8k9rr\""
Jul 28 01:14:44.278893 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.278870    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 01:14:44.283819 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.283797    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"]
Jul 28 01:14:44.316663 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.316643    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/01f02fd0-c957-42e1-bb0f-4481cf6c30b5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dhxcx\" (UID: \"01f02fd0-c957-42e1-bb0f-4481cf6c30b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"
Jul 28 01:14:44.316755 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.316670    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/01f02fd0-c957-42e1-bb0f-4481cf6c30b5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dhxcx\" (UID: \"01f02fd0-c957-42e1-bb0f-4481cf6c30b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"
Jul 28 01:14:44.316802 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.316778    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfllh\" (UniqueName: \"kubernetes.io/projected/01f02fd0-c957-42e1-bb0f-4481cf6c30b5-kube-api-access-qfllh\") pod \"kuadrant-console-plugin-66b5774985-dhxcx\" (UID: \"01f02fd0-c957-42e1-bb0f-4481cf6c30b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"
Jul 28 01:14:44.418208 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.418182    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/01f02fd0-c957-42e1-bb0f-4481cf6c30b5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dhxcx\" (UID: \"01f02fd0-c957-42e1-bb0f-4481cf6c30b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"
Jul 28 01:14:44.418403 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.418219    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/01f02fd0-c957-42e1-bb0f-4481cf6c30b5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dhxcx\" (UID: \"01f02fd0-c957-42e1-bb0f-4481cf6c30b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"
Jul 28 01:14:44.418403 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:14:44.418338    2565 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 28 01:14:44.418403 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.418355    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfllh\" (UniqueName: \"kubernetes.io/projected/01f02fd0-c957-42e1-bb0f-4481cf6c30b5-kube-api-access-qfllh\") pod \"kuadrant-console-plugin-66b5774985-dhxcx\" (UID: \"01f02fd0-c957-42e1-bb0f-4481cf6c30b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"
Jul 28 01:14:44.418605 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:14:44.418438    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01f02fd0-c957-42e1-bb0f-4481cf6c30b5-plugin-serving-cert podName:01f02fd0-c957-42e1-bb0f-4481cf6c30b5 nodeName:}" failed. No retries permitted until 2026-07-28 01:14:44.918416383 +0000 UTC m=+635.963310336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/01f02fd0-c957-42e1-bb0f-4481cf6c30b5-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-dhxcx" (UID: "01f02fd0-c957-42e1-bb0f-4481cf6c30b5") : secret "plugin-serving-cert" not found
Jul 28 01:14:44.418812 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.418792    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/01f02fd0-c957-42e1-bb0f-4481cf6c30b5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dhxcx\" (UID: \"01f02fd0-c957-42e1-bb0f-4481cf6c30b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"
Jul 28 01:14:44.426975 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.426948    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfllh\" (UniqueName: \"kubernetes.io/projected/01f02fd0-c957-42e1-bb0f-4481cf6c30b5-kube-api-access-qfllh\") pod \"kuadrant-console-plugin-66b5774985-dhxcx\" (UID: \"01f02fd0-c957-42e1-bb0f-4481cf6c30b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"
Jul 28 01:14:44.922117 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.922086    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/01f02fd0-c957-42e1-bb0f-4481cf6c30b5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dhxcx\" (UID: \"01f02fd0-c957-42e1-bb0f-4481cf6c30b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"
Jul 28 01:14:44.924541 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:44.924520    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/01f02fd0-c957-42e1-bb0f-4481cf6c30b5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dhxcx\" (UID: \"01f02fd0-c957-42e1-bb0f-4481cf6c30b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"
Jul 28 01:14:45.183328 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:45.183247    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"
Jul 28 01:14:45.310595 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:45.310571    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx"]
Jul 28 01:14:45.312724 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:14:45.312698    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01f02fd0_c957_42e1_bb0f_4481cf6c30b5.slice/crio-80f5eeb374a5f8b7bada589dd7158ddd907eb5776f30d1ca88b63d157c1ab56e WatchSource:0}: Error finding container 80f5eeb374a5f8b7bada589dd7158ddd907eb5776f30d1ca88b63d157c1ab56e: Status 404 returned error can't find the container with id 80f5eeb374a5f8b7bada589dd7158ddd907eb5776f30d1ca88b63d157c1ab56e
Jul 28 01:14:45.871257 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:45.871224    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx" event={"ID":"01f02fd0-c957-42e1-bb0f-4481cf6c30b5","Type":"ContainerStarted","Data":"80f5eeb374a5f8b7bada589dd7158ddd907eb5776f30d1ca88b63d157c1ab56e"}
Jul 28 01:14:54.204743 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:54.204664    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"]
Jul 28 01:14:54.211979 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:54.211956    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"
Jul 28 01:14:54.216371 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:54.216153    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9zxl2\""
Jul 28 01:14:54.224907 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:54.224881    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"]
Jul 28 01:14:54.309503 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:54.309450    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e45b21e3-24a4-4acf-a816-6f707b7401c7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-684bk\" (UID: \"e45b21e3-24a4-4acf-a816-6f707b7401c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"
Jul 28 01:14:54.309678 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:54.309531    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl6fg\" (UniqueName: \"kubernetes.io/projected/e45b21e3-24a4-4acf-a816-6f707b7401c7-kube-api-access-tl6fg\") pod \"kuadrant-operator-controller-manager-86878c769f-684bk\" (UID: \"e45b21e3-24a4-4acf-a816-6f707b7401c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"
Jul 28 01:14:54.410284 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:54.410245    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e45b21e3-24a4-4acf-a816-6f707b7401c7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-684bk\" (UID: \"e45b21e3-24a4-4acf-a816-6f707b7401c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"
Jul 28 01:14:54.410476 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:54.410294    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl6fg\" (UniqueName: \"kubernetes.io/projected/e45b21e3-24a4-4acf-a816-6f707b7401c7-kube-api-access-tl6fg\") pod \"kuadrant-operator-controller-manager-86878c769f-684bk\" (UID: \"e45b21e3-24a4-4acf-a816-6f707b7401c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"
Jul 28 01:14:54.410705 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:54.410682    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e45b21e3-24a4-4acf-a816-6f707b7401c7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-684bk\" (UID: \"e45b21e3-24a4-4acf-a816-6f707b7401c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"
Jul 28 01:14:54.423123 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:54.423092    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl6fg\" (UniqueName: \"kubernetes.io/projected/e45b21e3-24a4-4acf-a816-6f707b7401c7-kube-api-access-tl6fg\") pod \"kuadrant-operator-controller-manager-86878c769f-684bk\" (UID: \"e45b21e3-24a4-4acf-a816-6f707b7401c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"
Jul 28 01:14:54.525020 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:14:54.524936    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"
Jul 28 01:15:08.002296 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:15:08.002266    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"]
Jul 28 01:15:08.004462 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:15:08.004436    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45b21e3_24a4_4acf_a816_6f707b7401c7.slice/crio-f22076f883aaa7bcbde67162e4ef0cf4ac58cfc5d18ea1714a41077bb4df3565 WatchSource:0}: Error finding container f22076f883aaa7bcbde67162e4ef0cf4ac58cfc5d18ea1714a41077bb4df3565: Status 404 returned error can't find the container with id f22076f883aaa7bcbde67162e4ef0cf4ac58cfc5d18ea1714a41077bb4df3565
Jul 28 01:15:08.995716 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:15:08.995684    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk" event={"ID":"e45b21e3-24a4-4acf-a816-6f707b7401c7","Type":"ContainerStarted","Data":"f22076f883aaa7bcbde67162e4ef0cf4ac58cfc5d18ea1714a41077bb4df3565"}
Jul 28 01:15:08.997093 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:15:08.997069    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx" event={"ID":"01f02fd0-c957-42e1-bb0f-4481cf6c30b5","Type":"ContainerStarted","Data":"db5a6c54aecaf1ebc54068ae9ef688b94a21ef7253152c8142ada705927f6a21"}
Jul 28 01:15:09.014717 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:15:09.014672    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dhxcx" podStartSLOduration=2.075685986 podStartE2EDuration="25.014655928s" totalImagesPullingTime="22.938969942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:14:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:15:09.013460264 +0000 UTC m=+660.058354230" watchObservedRunningTime="2026-07-28 01:15:09.014655928 +0000 UTC m=+660.059549896"
Jul 28 01:15:14.022905 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:15:14.022868    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk" event={"ID":"e45b21e3-24a4-4acf-a816-6f707b7401c7","Type":"ContainerStarted","Data":"2377c9fe56b52e3311c65d6364e4f2e42c28939b2b9cf4aa252f3fca5b01d316"}
Jul 28 01:15:14.023368 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:15:14.022994    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"
Jul 28 01:15:14.043857 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:15:14.043812    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk" podStartSLOduration=14.590747598 podStartE2EDuration="20.043785442s" totalImagesPullingTime="5.453037844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:15:14.041537154 +0000 UTC m=+665.086431122" watchObservedRunningTime="2026-07-28 01:15:14.043785442 +0000 UTC m=+665.088679409"
Jul 28 01:15:25.029077 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:15:25.029045    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"
Jul 28 01:16:23.589278 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:23.589243    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-686c864877-ln67g"]
Jul 28 01:16:23.591996 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:23.591979    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686c864877-ln67g"
Jul 28 01:16:23.595004 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:23.594981    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 28 01:16:23.595252 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:23.595226    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-22sxl\""
Jul 28 01:16:23.596156 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:23.596137    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 28 01:16:23.600690 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:23.600667    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-686c864877-ln67g"]
Jul 28 01:16:23.673822 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:23.673790    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgct7\" (UniqueName: \"kubernetes.io/projected/d1d4c585-071a-4515-9124-e7645de15421-kube-api-access-hgct7\") pod \"maas-controller-686c864877-ln67g\" (UID: \"d1d4c585-071a-4515-9124-e7645de15421\") " pod="opendatahub/maas-controller-686c864877-ln67g"
Jul 28 01:16:23.673990 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:23.673846    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d1d4c585-071a-4515-9124-e7645de15421-webhook-cert\") pod \"maas-controller-686c864877-ln67g\" (UID: \"d1d4c585-071a-4515-9124-e7645de15421\") " pod="opendatahub/maas-controller-686c864877-ln67g"
Jul 28 01:16:23.774886 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:23.774841    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgct7\" (UniqueName: \"kubernetes.io/projected/d1d4c585-071a-4515-9124-e7645de15421-kube-api-access-hgct7\") pod \"maas-controller-686c864877-ln67g\" (UID: \"d1d4c585-071a-4515-9124-e7645de15421\") " pod="opendatahub/maas-controller-686c864877-ln67g"
Jul 28 01:16:23.775065 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:23.774898    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d1d4c585-071a-4515-9124-e7645de15421-webhook-cert\") pod \"maas-controller-686c864877-ln67g\" (UID: \"d1d4c585-071a-4515-9124-e7645de15421\") " pod="opendatahub/maas-controller-686c864877-ln67g"
Jul 28 01:16:23.775137 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:16:23.775064    2565 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 28 01:16:23.775200 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:16:23.775149    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1d4c585-071a-4515-9124-e7645de15421-webhook-cert podName:d1d4c585-071a-4515-9124-e7645de15421 nodeName:}" failed. No retries permitted until 2026-07-28 01:16:24.27512823 +0000 UTC m=+735.320022180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/d1d4c585-071a-4515-9124-e7645de15421-webhook-cert") pod "maas-controller-686c864877-ln67g" (UID: "d1d4c585-071a-4515-9124-e7645de15421") : secret "maas-controller-webhook-cert" not found
Jul 28 01:16:23.783836 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:23.783811    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgct7\" (UniqueName: \"kubernetes.io/projected/d1d4c585-071a-4515-9124-e7645de15421-kube-api-access-hgct7\") pod \"maas-controller-686c864877-ln67g\" (UID: \"d1d4c585-071a-4515-9124-e7645de15421\") " pod="opendatahub/maas-controller-686c864877-ln67g"
Jul 28 01:16:24.279345 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:24.279310    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d1d4c585-071a-4515-9124-e7645de15421-webhook-cert\") pod \"maas-controller-686c864877-ln67g\" (UID: \"d1d4c585-071a-4515-9124-e7645de15421\") " pod="opendatahub/maas-controller-686c864877-ln67g"
Jul 28 01:16:24.281914 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:24.281893    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d1d4c585-071a-4515-9124-e7645de15421-webhook-cert\") pod \"maas-controller-686c864877-ln67g\" (UID: \"d1d4c585-071a-4515-9124-e7645de15421\") " pod="opendatahub/maas-controller-686c864877-ln67g"
Jul 28 01:16:24.504771 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:24.504733    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686c864877-ln67g"
Jul 28 01:16:24.635990 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:24.635962    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-686c864877-ln67g"]
Jul 28 01:16:24.637970 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:16:24.637935    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d4c585_071a_4515_9124_e7645de15421.slice/crio-03ac2827a9d005f20a9dd14dd5278a6cea59f69ceab5d254d1abf2e37de94bf5 WatchSource:0}: Error finding container 03ac2827a9d005f20a9dd14dd5278a6cea59f69ceab5d254d1abf2e37de94bf5: Status 404 returned error can't find the container with id 03ac2827a9d005f20a9dd14dd5278a6cea59f69ceab5d254d1abf2e37de94bf5
Jul 28 01:16:24.639302 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:24.639282    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:16:25.356132 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:25.356026    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686c864877-ln67g" event={"ID":"d1d4c585-071a-4515-9124-e7645de15421","Type":"ContainerStarted","Data":"03ac2827a9d005f20a9dd14dd5278a6cea59f69ceab5d254d1abf2e37de94bf5"}
Jul 28 01:16:27.367767 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:27.367732    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686c864877-ln67g" event={"ID":"d1d4c585-071a-4515-9124-e7645de15421","Type":"ContainerStarted","Data":"bbd5eeef25bc461a9ea859f132721d6ac9d60703a36647912e05dfa67e1c95f1"}
Jul 28 01:16:27.368171 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:27.367837    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-686c864877-ln67g"
Jul 28 01:16:27.385717 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:27.385673    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-686c864877-ln67g" podStartSLOduration=2.317895131 podStartE2EDuration="4.385659793s" totalImagesPullingTime="2.067764662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:16:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:16:27.383594412 +0000 UTC m=+738.428488377" watchObservedRunningTime="2026-07-28 01:16:27.385659793 +0000 UTC m=+738.430553759"
Jul 28 01:16:38.176049 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.176017    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-686c864877-ln67g"]
Jul 28 01:16:38.176438 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.176214    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-686c864877-ln67g" podUID="d1d4c585-071a-4515-9124-e7645de15421" containerName="manager" containerID="cri-o://bbd5eeef25bc461a9ea859f132721d6ac9d60703a36647912e05dfa67e1c95f1" gracePeriod=30
Jul 28 01:16:38.177760 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.177735    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-686c864877-ln67g"
Jul 28 01:16:38.417831 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.417809    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686c864877-ln67g"
Jul 28 01:16:38.418681 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.418656    2565 generic.go:356] "Generic (PLEG): container finished" podID="d1d4c585-071a-4515-9124-e7645de15421" containerID="bbd5eeef25bc461a9ea859f132721d6ac9d60703a36647912e05dfa67e1c95f1" exitCode=0
Jul 28 01:16:38.418786 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.418735    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686c864877-ln67g" event={"ID":"d1d4c585-071a-4515-9124-e7645de15421","Type":"ContainerDied","Data":"bbd5eeef25bc461a9ea859f132721d6ac9d60703a36647912e05dfa67e1c95f1"}
Jul 28 01:16:38.418838 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.418785    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-686c864877-ln67g" event={"ID":"d1d4c585-071a-4515-9124-e7645de15421","Type":"ContainerDied","Data":"03ac2827a9d005f20a9dd14dd5278a6cea59f69ceab5d254d1abf2e37de94bf5"}
Jul 28 01:16:38.418838 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.418809    2565 scope.go:117] "RemoveContainer" containerID="bbd5eeef25bc461a9ea859f132721d6ac9d60703a36647912e05dfa67e1c95f1"
Jul 28 01:16:38.426667 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.426649    2565 scope.go:117] "RemoveContainer" containerID="bbd5eeef25bc461a9ea859f132721d6ac9d60703a36647912e05dfa67e1c95f1"
Jul 28 01:16:38.426928 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:16:38.426907    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbd5eeef25bc461a9ea859f132721d6ac9d60703a36647912e05dfa67e1c95f1\": container with ID starting with bbd5eeef25bc461a9ea859f132721d6ac9d60703a36647912e05dfa67e1c95f1 not found: ID does not exist" containerID="bbd5eeef25bc461a9ea859f132721d6ac9d60703a36647912e05dfa67e1c95f1"
Jul 28 01:16:38.426983 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.426934    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbd5eeef25bc461a9ea859f132721d6ac9d60703a36647912e05dfa67e1c95f1"} err="failed to get container status \"bbd5eeef25bc461a9ea859f132721d6ac9d60703a36647912e05dfa67e1c95f1\": rpc error: code = NotFound desc = could not find container \"bbd5eeef25bc461a9ea859f132721d6ac9d60703a36647912e05dfa67e1c95f1\": container with ID starting with bbd5eeef25bc461a9ea859f132721d6ac9d60703a36647912e05dfa67e1c95f1 not found: ID does not exist"
Jul 28 01:16:38.501496 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.501456    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d1d4c585-071a-4515-9124-e7645de15421-webhook-cert\") pod \"d1d4c585-071a-4515-9124-e7645de15421\" (UID: \"d1d4c585-071a-4515-9124-e7645de15421\") "
Jul 28 01:16:38.501631 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.501528    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgct7\" (UniqueName: \"kubernetes.io/projected/d1d4c585-071a-4515-9124-e7645de15421-kube-api-access-hgct7\") pod \"d1d4c585-071a-4515-9124-e7645de15421\" (UID: \"d1d4c585-071a-4515-9124-e7645de15421\") "
Jul 28 01:16:38.503738 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.503710    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1d4c585-071a-4515-9124-e7645de15421-kube-api-access-hgct7" (OuterVolumeSpecName: "kube-api-access-hgct7") pod "d1d4c585-071a-4515-9124-e7645de15421" (UID: "d1d4c585-071a-4515-9124-e7645de15421"). InnerVolumeSpecName "kube-api-access-hgct7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:16:38.503831 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.503732    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1d4c585-071a-4515-9124-e7645de15421-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d1d4c585-071a-4515-9124-e7645de15421" (UID: "d1d4c585-071a-4515-9124-e7645de15421"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:16:38.603065 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.603035    2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d1d4c585-071a-4515-9124-e7645de15421-webhook-cert\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:16:38.603065 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:38.603066    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgct7\" (UniqueName: \"kubernetes.io/projected/d1d4c585-071a-4515-9124-e7645de15421-kube-api-access-hgct7\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:16:39.424031 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:39.423998    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-686c864877-ln67g"
Jul 28 01:16:39.444305 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:39.444284    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-686c864877-ln67g"]
Jul 28 01:16:39.445759 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:39.445741    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-686c864877-ln67g"]
Jul 28 01:16:39.466333 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:16:39.466306    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1d4c585-071a-4515-9124-e7645de15421" path="/var/lib/kubelet/pods/d1d4c585-071a-4515-9124-e7645de15421/volumes"
Jul 28 01:17:00.023910 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.023866    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"]
Jul 28 01:17:00.024401 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.024221    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1d4c585-071a-4515-9124-e7645de15421" containerName="manager"
Jul 28 01:17:00.024401 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.024233    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1d4c585-071a-4515-9124-e7645de15421" containerName="manager"
Jul 28 01:17:00.024401 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.024306    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1d4c585-071a-4515-9124-e7645de15421" containerName="manager"
Jul 28 01:17:00.026847 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.026831    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"
Jul 28 01:17:00.029887 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.029863    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 28 01:17:00.030012 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.029897    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qcmwf\""
Jul 28 01:17:00.030154 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.030138    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:17:00.031106 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.031088    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 01:17:00.038166 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.038145    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"]
Jul 28 01:17:00.085853 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.085812    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b57dd6cc-85a4-4168-904f-8640c67fef7d-openshift-service-ca\") pod \"maas-api-59df996f58-q8ndp\" (UID: \"b57dd6cc-85a4-4168-904f-8640c67fef7d\") " pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"
Jul 28 01:17:00.086020 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.085902    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b57dd6cc-85a4-4168-904f-8640c67fef7d-maas-api-tls\") pod \"maas-api-59df996f58-q8ndp\" (UID: \"b57dd6cc-85a4-4168-904f-8640c67fef7d\") " pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"
Jul 28 01:17:00.086075 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.086015    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmq9p\" (UniqueName: \"kubernetes.io/projected/b57dd6cc-85a4-4168-904f-8640c67fef7d-kube-api-access-pmq9p\") pod \"maas-api-59df996f58-q8ndp\" (UID: \"b57dd6cc-85a4-4168-904f-8640c67fef7d\") " pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"
Jul 28 01:17:00.187113 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.187083    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b57dd6cc-85a4-4168-904f-8640c67fef7d-openshift-service-ca\") pod \"maas-api-59df996f58-q8ndp\" (UID: \"b57dd6cc-85a4-4168-904f-8640c67fef7d\") " pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"
Jul 28 01:17:00.187303 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.187135    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b57dd6cc-85a4-4168-904f-8640c67fef7d-maas-api-tls\") pod \"maas-api-59df996f58-q8ndp\" (UID: \"b57dd6cc-85a4-4168-904f-8640c67fef7d\") " pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"
Jul 28 01:17:00.187303 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.187187    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmq9p\" (UniqueName: \"kubernetes.io/projected/b57dd6cc-85a4-4168-904f-8640c67fef7d-kube-api-access-pmq9p\") pod \"maas-api-59df996f58-q8ndp\" (UID: \"b57dd6cc-85a4-4168-904f-8640c67fef7d\") " pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"
Jul 28 01:17:00.187428 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:17:00.187312    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 28 01:17:00.187428 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:17:00.187408    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57dd6cc-85a4-4168-904f-8640c67fef7d-maas-api-tls podName:b57dd6cc-85a4-4168-904f-8640c67fef7d nodeName:}" failed. No retries permitted until 2026-07-28 01:17:00.687386412 +0000 UTC m=+771.732280366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b57dd6cc-85a4-4168-904f-8640c67fef7d-maas-api-tls") pod "maas-api-59df996f58-q8ndp" (UID: "b57dd6cc-85a4-4168-904f-8640c67fef7d") : secret "maas-api-serving-cert" not found
Jul 28 01:17:00.187836 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.187814    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b57dd6cc-85a4-4168-904f-8640c67fef7d-openshift-service-ca\") pod \"maas-api-59df996f58-q8ndp\" (UID: \"b57dd6cc-85a4-4168-904f-8640c67fef7d\") " pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"
Jul 28 01:17:00.195722 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.195701    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmq9p\" (UniqueName: \"kubernetes.io/projected/b57dd6cc-85a4-4168-904f-8640c67fef7d-kube-api-access-pmq9p\") pod \"maas-api-59df996f58-q8ndp\" (UID: \"b57dd6cc-85a4-4168-904f-8640c67fef7d\") " pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"
Jul 28 01:17:00.693501 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.693457    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b57dd6cc-85a4-4168-904f-8640c67fef7d-maas-api-tls\") pod \"maas-api-59df996f58-q8ndp\" (UID: \"b57dd6cc-85a4-4168-904f-8640c67fef7d\") " pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"
Jul 28 01:17:00.696091 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.696070    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b57dd6cc-85a4-4168-904f-8640c67fef7d-maas-api-tls\") pod \"maas-api-59df996f58-q8ndp\" (UID: \"b57dd6cc-85a4-4168-904f-8640c67fef7d\") " pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"
Jul 28 01:17:00.938799 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:00.938765    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"
Jul 28 01:17:01.119685 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:01.119661    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"]
Jul 28 01:17:01.121905 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:17:01.121876    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb57dd6cc_85a4_4168_904f_8640c67fef7d.slice/crio-2ad349609ce3419e915c19349b3ddf82074896ebad53a3be944bf0baa69b475b WatchSource:0}: Error finding container 2ad349609ce3419e915c19349b3ddf82074896ebad53a3be944bf0baa69b475b: Status 404 returned error can't find the container with id 2ad349609ce3419e915c19349b3ddf82074896ebad53a3be944bf0baa69b475b
Jul 28 01:17:01.524512 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:01.524465    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp" event={"ID":"b57dd6cc-85a4-4168-904f-8640c67fef7d","Type":"ContainerStarted","Data":"2ad349609ce3419e915c19349b3ddf82074896ebad53a3be944bf0baa69b475b"}
Jul 28 01:17:03.535005 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:03.534967    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp" event={"ID":"b57dd6cc-85a4-4168-904f-8640c67fef7d","Type":"ContainerStarted","Data":"bd3a49e5b59693420194fa25dfae3eab5ce41911c8859bd3f22054bcf6805523"}
Jul 28 01:17:03.535366 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:03.535043    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"
Jul 28 01:17:03.551643 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:03.551592    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp" podStartSLOduration=1.620455313 podStartE2EDuration="3.551578635s" totalImagesPullingTime="1.931123322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:17:03.549451575 +0000 UTC m=+774.594345544" watchObservedRunningTime="2026-07-28 01:17:03.551578635 +0000 UTC m=+774.596472601"
Jul 28 01:17:09.545055 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:09.545024    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59df996f58-q8ndp"
Jul 28 01:17:56.617644 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.617610    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"]
Jul 28 01:17:56.621644 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.621625    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.625841 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.625819    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 28 01:17:56.625944 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.625821    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 01:17:56.625944 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.625821    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bb82g\""
Jul 28 01:17:56.625944 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.625822    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 01:17:56.634244 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.634225    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"]
Jul 28 01:17:56.680546 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.680521    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9857088a-a053-4582-9e7e-faa87bf4f972-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.680634 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.680564    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9857088a-a053-4582-9e7e-faa87bf4f972-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.680634 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.680608    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9857088a-a053-4582-9e7e-faa87bf4f972-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.680716 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.680640    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9857088a-a053-4582-9e7e-faa87bf4f972-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.680749 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.680736    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9857088a-a053-4582-9e7e-faa87bf4f972-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.680785 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.680761    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xh8g\" (UniqueName: \"kubernetes.io/projected/9857088a-a053-4582-9e7e-faa87bf4f972-kube-api-access-8xh8g\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.782038 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.782005    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9857088a-a053-4582-9e7e-faa87bf4f972-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.782184 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.782056    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9857088a-a053-4582-9e7e-faa87bf4f972-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.782184 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.782111    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9857088a-a053-4582-9e7e-faa87bf4f972-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.782184 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.782133    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xh8g\" (UniqueName: \"kubernetes.io/projected/9857088a-a053-4582-9e7e-faa87bf4f972-kube-api-access-8xh8g\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.782184 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.782173    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9857088a-a053-4582-9e7e-faa87bf4f972-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.782374 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.782220    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9857088a-a053-4582-9e7e-faa87bf4f972-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.782531 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.782504    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9857088a-a053-4582-9e7e-faa87bf4f972-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.782622 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.782531    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9857088a-a053-4582-9e7e-faa87bf4f972-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.782622 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.782569    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9857088a-a053-4582-9e7e-faa87bf4f972-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.784466 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.784448    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9857088a-a053-4582-9e7e-faa87bf4f972-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.784636 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.784620    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9857088a-a053-4582-9e7e-faa87bf4f972-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.790612 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.790589    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xh8g\" (UniqueName: \"kubernetes.io/projected/9857088a-a053-4582-9e7e-faa87bf4f972-kube-api-access-8xh8g\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k\" (UID: \"9857088a-a053-4582-9e7e-faa87bf4f972\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:56.931985 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:56.931900    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:17:57.063843 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:57.063816    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"]
Jul 28 01:17:57.065304 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:17:57.065278    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9857088a_a053_4582_9e7e_faa87bf4f972.slice/crio-23d79cc87e28b92b029d3e7d2e8cf7421d29159fbb55341499611e1d888ab9b3 WatchSource:0}: Error finding container 23d79cc87e28b92b029d3e7d2e8cf7421d29159fbb55341499611e1d888ab9b3: Status 404 returned error can't find the container with id 23d79cc87e28b92b029d3e7d2e8cf7421d29159fbb55341499611e1d888ab9b3
Jul 28 01:17:57.778256 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:57.778220    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k" event={"ID":"9857088a-a053-4582-9e7e-faa87bf4f972","Type":"ContainerStarted","Data":"23d79cc87e28b92b029d3e7d2e8cf7421d29159fbb55341499611e1d888ab9b3"}
Jul 28 01:17:58.784224 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:17:58.784188    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k" event={"ID":"9857088a-a053-4582-9e7e-faa87bf4f972","Type":"ContainerStarted","Data":"c8403f09807abffe1b5c5281b1371e0320590b9bbb9904bdf8cf71785b04e523"}
Jul 28 01:18:06.932458 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:06.932428    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:18:06.932915 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:06.932744    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:18:06.945027 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:06.944999    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:18:06.964210 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:06.964159    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k" podStartSLOduration=9.911709133 podStartE2EDuration="10.964148537s" totalImagesPullingTime="1.052439404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:17:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:17:58.80287002 +0000 UTC m=+829.847763987" watchObservedRunningTime="2026-07-28 01:18:06.964148537 +0000 UTC m=+838.009042505"
Jul 28 01:18:07.837052 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:07.837019    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k"
Jul 28 01:18:08.522536 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.522506    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"]
Jul 28 01:18:08.525655 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.525639    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.528207 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.528185    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 28 01:18:08.538968 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.538948    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"]
Jul 28 01:18:08.586932 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.586905    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6120e58f-7665-4c20-a550-4a57e1b34ad9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.587046 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.586962    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6120e58f-7665-4c20-a550-4a57e1b34ad9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.587046 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.587006    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6120e58f-7665-4c20-a550-4a57e1b34ad9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.587124 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.587062    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6120e58f-7665-4c20-a550-4a57e1b34ad9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.587124 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.587086    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbqdr\" (UniqueName: \"kubernetes.io/projected/6120e58f-7665-4c20-a550-4a57e1b34ad9-kube-api-access-qbqdr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.587199 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.587165    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6120e58f-7665-4c20-a550-4a57e1b34ad9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.687827 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.687793    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6120e58f-7665-4c20-a550-4a57e1b34ad9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.687827 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.687824    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6120e58f-7665-4c20-a550-4a57e1b34ad9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.688004 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.687855    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6120e58f-7665-4c20-a550-4a57e1b34ad9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.688004 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.687884    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbqdr\" (UniqueName: \"kubernetes.io/projected/6120e58f-7665-4c20-a550-4a57e1b34ad9-kube-api-access-qbqdr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.688004 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.687932    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6120e58f-7665-4c20-a550-4a57e1b34ad9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.688004 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.687962    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6120e58f-7665-4c20-a550-4a57e1b34ad9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.688251 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.688212    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6120e58f-7665-4c20-a550-4a57e1b34ad9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.688251 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.688239    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6120e58f-7665-4c20-a550-4a57e1b34ad9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.688396 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.688371    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6120e58f-7665-4c20-a550-4a57e1b34ad9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.690111 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.690091    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6120e58f-7665-4c20-a550-4a57e1b34ad9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.690343 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.690325    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6120e58f-7665-4c20-a550-4a57e1b34ad9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.697631 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.697610    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbqdr\" (UniqueName: \"kubernetes.io/projected/6120e58f-7665-4c20-a550-4a57e1b34ad9-kube-api-access-qbqdr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rxpcw\" (UID: \"6120e58f-7665-4c20-a550-4a57e1b34ad9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.835685 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.835657    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:08.970687 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:08.970660    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"]
Jul 28 01:18:08.972354 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:18:08.972329    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6120e58f_7665_4c20_a550_4a57e1b34ad9.slice/crio-0c6fdcd41104bb62c3f1d8e3f2a85819f7bde9560f14a63a11a37d6d8deac42a WatchSource:0}: Error finding container 0c6fdcd41104bb62c3f1d8e3f2a85819f7bde9560f14a63a11a37d6d8deac42a: Status 404 returned error can't find the container with id 0c6fdcd41104bb62c3f1d8e3f2a85819f7bde9560f14a63a11a37d6d8deac42a
Jul 28 01:18:09.836745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:09.836710    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw" event={"ID":"6120e58f-7665-4c20-a550-4a57e1b34ad9","Type":"ContainerStarted","Data":"ca1b7e6d9662644e8ea73bf53d20ac796e219f58bf0e1893ba59f532db6298b0"}
Jul 28 01:18:09.837102 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:09.836765    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw" event={"ID":"6120e58f-7665-4c20-a550-4a57e1b34ad9","Type":"ContainerStarted","Data":"0c6fdcd41104bb62c3f1d8e3f2a85819f7bde9560f14a63a11a37d6d8deac42a"}
Jul 28 01:18:09.854865 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:09.854822    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw" podStartSLOduration=1.641382595 podStartE2EDuration="1.854809807s" totalImagesPullingTime="213.427212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:18:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:18:09.853633263 +0000 UTC m=+840.898527232" watchObservedRunningTime="2026-07-28 01:18:09.854809807 +0000 UTC m=+840.899703775"
Jul 28 01:18:12.313189 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.313156    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"]
Jul 28 01:18:12.316644 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.316622    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.319150 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.319131    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 28 01:18:12.326658 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.326624    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"]
Jul 28 01:18:12.422402 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.422371    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj2fr\" (UniqueName: \"kubernetes.io/projected/f3b10cf3-4750-460e-bda7-783e4da98f0b-kube-api-access-jj2fr\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.422588 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.422417    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3b10cf3-4750-460e-bda7-783e4da98f0b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.422588 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.422553    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3b10cf3-4750-460e-bda7-783e4da98f0b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.422662 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.422596    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3b10cf3-4750-460e-bda7-783e4da98f0b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.422662 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.422614    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3b10cf3-4750-460e-bda7-783e4da98f0b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.422724 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.422663    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3b10cf3-4750-460e-bda7-783e4da98f0b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.523365 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.523327    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj2fr\" (UniqueName: \"kubernetes.io/projected/f3b10cf3-4750-460e-bda7-783e4da98f0b-kube-api-access-jj2fr\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.523550 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.523373    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3b10cf3-4750-460e-bda7-783e4da98f0b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.523597 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.523562    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3b10cf3-4750-460e-bda7-783e4da98f0b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.523641 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.523627    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3b10cf3-4750-460e-bda7-783e4da98f0b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.523679 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.523652    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3b10cf3-4750-460e-bda7-783e4da98f0b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.523731 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.523690    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3b10cf3-4750-460e-bda7-783e4da98f0b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.524045 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.524017    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3b10cf3-4750-460e-bda7-783e4da98f0b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.524141 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.524027    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3b10cf3-4750-460e-bda7-783e4da98f0b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.524141 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.524080    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3b10cf3-4750-460e-bda7-783e4da98f0b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.525696 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.525675    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3b10cf3-4750-460e-bda7-783e4da98f0b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.526003 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.525986    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3b10cf3-4750-460e-bda7-783e4da98f0b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.531020 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.531000    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj2fr\" (UniqueName: \"kubernetes.io/projected/f3b10cf3-4750-460e-bda7-783e4da98f0b-kube-api-access-jj2fr\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q\" (UID: \"f3b10cf3-4750-460e-bda7-783e4da98f0b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.626377 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.626301    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:12.755702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.755677    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"]
Jul 28 01:18:12.757558 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:18:12.757532    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3b10cf3_4750_460e_bda7_783e4da98f0b.slice/crio-6e23c0f9ef02b5d881dce540369dc23d9e7f05d3b28722c957c9ceae59a9d032 WatchSource:0}: Error finding container 6e23c0f9ef02b5d881dce540369dc23d9e7f05d3b28722c957c9ceae59a9d032: Status 404 returned error can't find the container with id 6e23c0f9ef02b5d881dce540369dc23d9e7f05d3b28722c957c9ceae59a9d032
Jul 28 01:18:12.849400 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:12.849365    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q" event={"ID":"f3b10cf3-4750-460e-bda7-783e4da98f0b","Type":"ContainerStarted","Data":"6e23c0f9ef02b5d881dce540369dc23d9e7f05d3b28722c957c9ceae59a9d032"}
Jul 28 01:18:13.856204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:13.856169    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q" event={"ID":"f3b10cf3-4750-460e-bda7-783e4da98f0b","Type":"ContainerStarted","Data":"03100a19ddd338cc3cbfd795e5b2661b02fce810fb6bb648b60dede2d82b5bb1"}
Jul 28 01:18:13.874811 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:13.874762    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q" podStartSLOduration=1.701320855 podStartE2EDuration="1.874747304s" totalImagesPullingTime="173.426449ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:18:13.873426832 +0000 UTC m=+844.918320799" watchObservedRunningTime="2026-07-28 01:18:13.874747304 +0000 UTC m=+844.919641586"
Jul 28 01:18:18.836538 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:18.836502    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:18.836935 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:18.836577    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:18.849050 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:18.849018    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:18.890163 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:18.890136    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rxpcw"
Jul 28 01:18:22.626605 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:22.626571    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:22.627012 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:22.626838    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:22.639043 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:22.639014    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:18:22.908718 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:18:22.908643    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q"
Jul 28 01:19:10.076016 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:19:10.075988    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:19:10.078322 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:19:10.077066    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:19:10.080067 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:19:10.080044    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:19:10.080990 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:19:10.080973    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:19:10.083616 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:19:10.083594    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:19:10.087362 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:19:10.087341    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:24:10.105688 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:24:10.105573    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:24:10.109561 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:24:10.106612    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:24:10.109561 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:24:10.108983    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:24:10.109888 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:24:10.109871    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:24:10.112707 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:24:10.112684    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:24:10.116140 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:24:10.116119    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:25:55.385676 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:55.385639    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"]
Jul 28 01:25:55.386175 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:55.385827    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk" podUID="e45b21e3-24a4-4acf-a816-6f707b7401c7" containerName="manager" containerID="cri-o://2377c9fe56b52e3311c65d6364e4f2e42c28939b2b9cf4aa252f3fca5b01d316" gracePeriod=10
Jul 28 01:25:58.837770 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:58.837745    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"
Jul 28 01:25:58.924754 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:58.924720    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e45b21e3-24a4-4acf-a816-6f707b7401c7-extensions-socket-volume\") pod \"e45b21e3-24a4-4acf-a816-6f707b7401c7\" (UID: \"e45b21e3-24a4-4acf-a816-6f707b7401c7\") "
Jul 28 01:25:58.924918 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:58.924808    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tl6fg\" (UniqueName: \"kubernetes.io/projected/e45b21e3-24a4-4acf-a816-6f707b7401c7-kube-api-access-tl6fg\") pod \"e45b21e3-24a4-4acf-a816-6f707b7401c7\" (UID: \"e45b21e3-24a4-4acf-a816-6f707b7401c7\") "
Jul 28 01:25:58.925072 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:58.925049    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45b21e3-24a4-4acf-a816-6f707b7401c7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e45b21e3-24a4-4acf-a816-6f707b7401c7" (UID: "e45b21e3-24a4-4acf-a816-6f707b7401c7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:25:58.927226 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:58.927170    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45b21e3-24a4-4acf-a816-6f707b7401c7-kube-api-access-tl6fg" (OuterVolumeSpecName: "kube-api-access-tl6fg") pod "e45b21e3-24a4-4acf-a816-6f707b7401c7" (UID: "e45b21e3-24a4-4acf-a816-6f707b7401c7"). InnerVolumeSpecName "kube-api-access-tl6fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:25:59.013475 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:59.013432    2565 generic.go:356] "Generic (PLEG): container finished" podID="e45b21e3-24a4-4acf-a816-6f707b7401c7" containerID="2377c9fe56b52e3311c65d6364e4f2e42c28939b2b9cf4aa252f3fca5b01d316" exitCode=0
Jul 28 01:25:59.013641 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:59.013518    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"
Jul 28 01:25:59.013641 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:59.013515    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk" event={"ID":"e45b21e3-24a4-4acf-a816-6f707b7401c7","Type":"ContainerDied","Data":"2377c9fe56b52e3311c65d6364e4f2e42c28939b2b9cf4aa252f3fca5b01d316"}
Jul 28 01:25:59.013641 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:59.013570    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk" event={"ID":"e45b21e3-24a4-4acf-a816-6f707b7401c7","Type":"ContainerDied","Data":"f22076f883aaa7bcbde67162e4ef0cf4ac58cfc5d18ea1714a41077bb4df3565"}
Jul 28 01:25:59.013641 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:59.013594    2565 scope.go:117] "RemoveContainer" containerID="2377c9fe56b52e3311c65d6364e4f2e42c28939b2b9cf4aa252f3fca5b01d316"
Jul 28 01:25:59.023039 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:59.023022    2565 scope.go:117] "RemoveContainer" containerID="2377c9fe56b52e3311c65d6364e4f2e42c28939b2b9cf4aa252f3fca5b01d316"
Jul 28 01:25:59.023310 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:25:59.023293    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2377c9fe56b52e3311c65d6364e4f2e42c28939b2b9cf4aa252f3fca5b01d316\": container with ID starting with 2377c9fe56b52e3311c65d6364e4f2e42c28939b2b9cf4aa252f3fca5b01d316 not found: ID does not exist" containerID="2377c9fe56b52e3311c65d6364e4f2e42c28939b2b9cf4aa252f3fca5b01d316"
Jul 28 01:25:59.023351 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:59.023319    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2377c9fe56b52e3311c65d6364e4f2e42c28939b2b9cf4aa252f3fca5b01d316"} err="failed to get container status \"2377c9fe56b52e3311c65d6364e4f2e42c28939b2b9cf4aa252f3fca5b01d316\": rpc error: code = NotFound desc = could not find container \"2377c9fe56b52e3311c65d6364e4f2e42c28939b2b9cf4aa252f3fca5b01d316\": container with ID starting with 2377c9fe56b52e3311c65d6364e4f2e42c28939b2b9cf4aa252f3fca5b01d316 not found: ID does not exist"
Jul 28 01:25:59.025602 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:59.025581    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e45b21e3-24a4-4acf-a816-6f707b7401c7-extensions-socket-volume\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:25:59.025602 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:59.025602    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl6fg\" (UniqueName: \"kubernetes.io/projected/e45b21e3-24a4-4acf-a816-6f707b7401c7-kube-api-access-tl6fg\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:25:59.037066 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:59.037046    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"]
Jul 28 01:25:59.043423 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:59.043403    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-684bk"]
Jul 28 01:25:59.466552 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:25:59.466509    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e45b21e3-24a4-4acf-a816-6f707b7401c7" path="/var/lib/kubelet/pods/e45b21e3-24a4-4acf-a816-6f707b7401c7/volumes"
Jul 28 01:26:56.563076 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:56.562980    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5"]
Jul 28 01:26:56.563618 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:56.563393    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e45b21e3-24a4-4acf-a816-6f707b7401c7" containerName="manager"
Jul 28 01:26:56.563618 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:56.563408    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e45b21e3-24a4-4acf-a816-6f707b7401c7" containerName="manager"
Jul 28 01:26:56.563618 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:56.563463    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e45b21e3-24a4-4acf-a816-6f707b7401c7" containerName="manager"
Jul 28 01:26:56.566345 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:56.566329    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5"
Jul 28 01:26:56.569287 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:56.569270    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9zxl2\""
Jul 28 01:26:56.585808 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:56.585786    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5"]
Jul 28 01:26:56.644577 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:56.644540    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-875sd\" (UniqueName: \"kubernetes.io/projected/864b86be-62aa-4635-804c-d56ef67e55f0-kube-api-access-875sd\") pod \"kuadrant-operator-controller-manager-86878c769f-j2mk5\" (UID: \"864b86be-62aa-4635-804c-d56ef67e55f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5"
Jul 28 01:26:56.644745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:56.644640    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/864b86be-62aa-4635-804c-d56ef67e55f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j2mk5\" (UID: \"864b86be-62aa-4635-804c-d56ef67e55f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5"
Jul 28 01:26:56.745412 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:56.745372    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/864b86be-62aa-4635-804c-d56ef67e55f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j2mk5\" (UID: \"864b86be-62aa-4635-804c-d56ef67e55f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5"
Jul 28 01:26:56.745599 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:56.745448    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-875sd\" (UniqueName: \"kubernetes.io/projected/864b86be-62aa-4635-804c-d56ef67e55f0-kube-api-access-875sd\") pod \"kuadrant-operator-controller-manager-86878c769f-j2mk5\" (UID: \"864b86be-62aa-4635-804c-d56ef67e55f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5"
Jul 28 01:26:56.745785 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:56.745767    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/864b86be-62aa-4635-804c-d56ef67e55f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-j2mk5\" (UID: \"864b86be-62aa-4635-804c-d56ef67e55f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5"
Jul 28 01:26:56.756459 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:56.756433    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-875sd\" (UniqueName: \"kubernetes.io/projected/864b86be-62aa-4635-804c-d56ef67e55f0-kube-api-access-875sd\") pod \"kuadrant-operator-controller-manager-86878c769f-j2mk5\" (UID: \"864b86be-62aa-4635-804c-d56ef67e55f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5"
Jul 28 01:26:56.876154 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:56.876067    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5"
Jul 28 01:26:57.025414 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:57.025386    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5"]
Jul 28 01:26:57.026199 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:26:57.026175    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod864b86be_62aa_4635_804c_d56ef67e55f0.slice/crio-a145f5f1c58c8b32840553382a663f318cb94da4da146e271ece60636ccd6e36 WatchSource:0}: Error finding container a145f5f1c58c8b32840553382a663f318cb94da4da146e271ece60636ccd6e36: Status 404 returned error can't find the container with id a145f5f1c58c8b32840553382a663f318cb94da4da146e271ece60636ccd6e36
Jul 28 01:26:57.279941 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:57.279846    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5" event={"ID":"864b86be-62aa-4635-804c-d56ef67e55f0","Type":"ContainerStarted","Data":"e059a8169299fd0085755c69d720e1bfa6c522bd22d891bcfb7c043a2372bd94"}
Jul 28 01:26:57.279941 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:57.279901    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5" event={"ID":"864b86be-62aa-4635-804c-d56ef67e55f0","Type":"ContainerStarted","Data":"a145f5f1c58c8b32840553382a663f318cb94da4da146e271ece60636ccd6e36"}
Jul 28 01:26:57.280131 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:57.279950    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5"
Jul 28 01:26:57.304310 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:26:57.304256    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5" podStartSLOduration=1.304238607 podStartE2EDuration="1.304238607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:26:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:26:57.301640321 +0000 UTC m=+1368.346534292" watchObservedRunningTime="2026-07-28 01:26:57.304238607 +0000 UTC m=+1368.349132575"
Jul 28 01:27:08.287237 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:27:08.287200    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-j2mk5"
Jul 28 01:29:10.134206 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:29:10.134093    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:29:10.138256 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:29:10.135014    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:29:10.139118 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:29:10.139101    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:29:10.140054 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:29:10.140039    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:29:10.140884 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:29:10.140867    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:29:10.145857 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:29:10.145843    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:34:01.633267 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:01.633178    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"]
Jul 28 01:34:01.636834 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:01.636813    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:01.640465 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:01.640436    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5ce185b9-serving-cert\""
Jul 28 01:34:01.652731 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:01.652706    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"]
Jul 28 01:34:01.710987 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:01.710956    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71244199-6298-46b6-8bac-7d0093b42aa2-maas-api-tls\") pod \"maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5\" (UID: \"71244199-6298-46b6-8bac-7d0093b42aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:01.711130 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:01.710991    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nhqr\" (UniqueName: \"kubernetes.io/projected/71244199-6298-46b6-8bac-7d0093b42aa2-kube-api-access-8nhqr\") pod \"maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5\" (UID: \"71244199-6298-46b6-8bac-7d0093b42aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:01.711130 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:01.711086    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71244199-6298-46b6-8bac-7d0093b42aa2-openshift-service-ca\") pod \"maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5\" (UID: \"71244199-6298-46b6-8bac-7d0093b42aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:01.811804 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:01.811762    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71244199-6298-46b6-8bac-7d0093b42aa2-openshift-service-ca\") pod \"maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5\" (UID: \"71244199-6298-46b6-8bac-7d0093b42aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:01.811995 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:01.811864    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71244199-6298-46b6-8bac-7d0093b42aa2-maas-api-tls\") pod \"maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5\" (UID: \"71244199-6298-46b6-8bac-7d0093b42aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:01.811995 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:01.811892    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nhqr\" (UniqueName: \"kubernetes.io/projected/71244199-6298-46b6-8bac-7d0093b42aa2-kube-api-access-8nhqr\") pod \"maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5\" (UID: \"71244199-6298-46b6-8bac-7d0093b42aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:01.812446 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:01.812421    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71244199-6298-46b6-8bac-7d0093b42aa2-openshift-service-ca\") pod \"maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5\" (UID: \"71244199-6298-46b6-8bac-7d0093b42aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:01.814440 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:01.814418    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71244199-6298-46b6-8bac-7d0093b42aa2-maas-api-tls\") pod \"maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5\" (UID: \"71244199-6298-46b6-8bac-7d0093b42aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:01.824292 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:01.824267    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nhqr\" (UniqueName: \"kubernetes.io/projected/71244199-6298-46b6-8bac-7d0093b42aa2-kube-api-access-8nhqr\") pod \"maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5\" (UID: \"71244199-6298-46b6-8bac-7d0093b42aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:01.949338 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:01.949252    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:02.295743 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:02.295718    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"]
Jul 28 01:34:02.298685 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:34:02.298659    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71244199_6298_46b6_8bac_7d0093b42aa2.slice/crio-adcf7bdb7e0d024c810dadf77e707d86f0f684be70609401a2fe6eab98920240 WatchSource:0}: Error finding container adcf7bdb7e0d024c810dadf77e707d86f0f684be70609401a2fe6eab98920240: Status 404 returned error can't find the container with id adcf7bdb7e0d024c810dadf77e707d86f0f684be70609401a2fe6eab98920240
Jul 28 01:34:02.300737 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:02.300711    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:34:03.261027 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:03.260932    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5" event={"ID":"71244199-6298-46b6-8bac-7d0093b42aa2","Type":"ContainerStarted","Data":"22cdfba06167dae7119e907541a17a0eead11686d33da5d2c42713c7e9dae2c6"}
Jul 28 01:34:03.261027 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:03.261010    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5" event={"ID":"71244199-6298-46b6-8bac-7d0093b42aa2","Type":"ContainerStarted","Data":"adcf7bdb7e0d024c810dadf77e707d86f0f684be70609401a2fe6eab98920240"}
Jul 28 01:34:03.261527 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:03.261042    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:03.288230 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:03.288179    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5" podStartSLOduration=1.700474082 podStartE2EDuration="2.288162782s" totalImagesPullingTime="587.6887ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:34:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:34:03.287297664 +0000 UTC m=+1794.332191633" watchObservedRunningTime="2026-07-28 01:34:03.288162782 +0000 UTC m=+1794.333056749"
Jul 28 01:34:09.270940 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:09.270910    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:10.163516 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:10.163393    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:34:10.174358 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:10.164415    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:34:10.174358 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:10.168318    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:34:10.174358 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:10.169292    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:34:10.174358 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:10.170868    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:34:10.176110 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:10.176094    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:34:42.553180 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:42.553149    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"]
Jul 28 01:34:42.553720 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:42.553345    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5" podUID="71244199-6298-46b6-8bac-7d0093b42aa2" containerName="maas-api" containerID="cri-o://22cdfba06167dae7119e907541a17a0eead11686d33da5d2c42713c7e9dae2c6" gracePeriod=30
Jul 28 01:34:42.801216 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:42.801195    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:42.908787 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:42.908760    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71244199-6298-46b6-8bac-7d0093b42aa2-maas-api-tls\") pod \"71244199-6298-46b6-8bac-7d0093b42aa2\" (UID: \"71244199-6298-46b6-8bac-7d0093b42aa2\") "
Jul 28 01:34:42.908955 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:42.908828    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nhqr\" (UniqueName: \"kubernetes.io/projected/71244199-6298-46b6-8bac-7d0093b42aa2-kube-api-access-8nhqr\") pod \"71244199-6298-46b6-8bac-7d0093b42aa2\" (UID: \"71244199-6298-46b6-8bac-7d0093b42aa2\") "
Jul 28 01:34:42.908955 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:42.908891    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71244199-6298-46b6-8bac-7d0093b42aa2-openshift-service-ca\") pod \"71244199-6298-46b6-8bac-7d0093b42aa2\" (UID: \"71244199-6298-46b6-8bac-7d0093b42aa2\") "
Jul 28 01:34:42.909230 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:42.909207    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71244199-6298-46b6-8bac-7d0093b42aa2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "71244199-6298-46b6-8bac-7d0093b42aa2" (UID: "71244199-6298-46b6-8bac-7d0093b42aa2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:34:42.911043 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:42.911019    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71244199-6298-46b6-8bac-7d0093b42aa2-kube-api-access-8nhqr" (OuterVolumeSpecName: "kube-api-access-8nhqr") pod "71244199-6298-46b6-8bac-7d0093b42aa2" (UID: "71244199-6298-46b6-8bac-7d0093b42aa2"). InnerVolumeSpecName "kube-api-access-8nhqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:34:42.911141 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:42.911104    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71244199-6298-46b6-8bac-7d0093b42aa2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "71244199-6298-46b6-8bac-7d0093b42aa2" (UID: "71244199-6298-46b6-8bac-7d0093b42aa2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:34:43.009908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:43.009875    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71244199-6298-46b6-8bac-7d0093b42aa2-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:34:43.009908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:43.009901    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71244199-6298-46b6-8bac-7d0093b42aa2-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:34:43.009908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:43.009910    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nhqr\" (UniqueName: \"kubernetes.io/projected/71244199-6298-46b6-8bac-7d0093b42aa2-kube-api-access-8nhqr\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:34:43.441994 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:43.441959    2565 generic.go:356] "Generic (PLEG): container finished" podID="71244199-6298-46b6-8bac-7d0093b42aa2" containerID="22cdfba06167dae7119e907541a17a0eead11686d33da5d2c42713c7e9dae2c6" exitCode=0
Jul 28 01:34:43.442170 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:43.442012    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"
Jul 28 01:34:43.442170 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:43.442050    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5" event={"ID":"71244199-6298-46b6-8bac-7d0093b42aa2","Type":"ContainerDied","Data":"22cdfba06167dae7119e907541a17a0eead11686d33da5d2c42713c7e9dae2c6"}
Jul 28 01:34:43.442170 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:43.442104    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5" event={"ID":"71244199-6298-46b6-8bac-7d0093b42aa2","Type":"ContainerDied","Data":"adcf7bdb7e0d024c810dadf77e707d86f0f684be70609401a2fe6eab98920240"}
Jul 28 01:34:43.442170 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:43.442126    2565 scope.go:117] "RemoveContainer" containerID="22cdfba06167dae7119e907541a17a0eead11686d33da5d2c42713c7e9dae2c6"
Jul 28 01:34:43.453352 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:43.453312    2565 scope.go:117] "RemoveContainer" containerID="22cdfba06167dae7119e907541a17a0eead11686d33da5d2c42713c7e9dae2c6"
Jul 28 01:34:43.453778 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:34:43.453757    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22cdfba06167dae7119e907541a17a0eead11686d33da5d2c42713c7e9dae2c6\": container with ID starting with 22cdfba06167dae7119e907541a17a0eead11686d33da5d2c42713c7e9dae2c6 not found: ID does not exist" containerID="22cdfba06167dae7119e907541a17a0eead11686d33da5d2c42713c7e9dae2c6"
Jul 28 01:34:43.453855 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:43.453787    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22cdfba06167dae7119e907541a17a0eead11686d33da5d2c42713c7e9dae2c6"} err="failed to get container status \"22cdfba06167dae7119e907541a17a0eead11686d33da5d2c42713c7e9dae2c6\": rpc error: code = NotFound desc = could not find container \"22cdfba06167dae7119e907541a17a0eead11686d33da5d2c42713c7e9dae2c6\": container with ID starting with 22cdfba06167dae7119e907541a17a0eead11686d33da5d2c42713c7e9dae2c6 not found: ID does not exist"
Jul 28 01:34:43.571006 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:43.570972    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"]
Jul 28 01:34:43.584561 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:43.584537    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5ce185b9-744c88d5fc-9pnt5"]
Jul 28 01:34:45.466829 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:45.466793    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71244199-6298-46b6-8bac-7d0093b42aa2" path="/var/lib/kubelet/pods/71244199-6298-46b6-8bac-7d0093b42aa2/volumes"
Jul 28 01:34:47.056443 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.056404    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"]
Jul 28 01:34:47.056961 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.056945    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71244199-6298-46b6-8bac-7d0093b42aa2" containerName="maas-api"
Jul 28 01:34:47.057002 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.056965    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="71244199-6298-46b6-8bac-7d0093b42aa2" containerName="maas-api"
Jul 28 01:34:47.057068 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.057057    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="71244199-6298-46b6-8bac-7d0093b42aa2" containerName="maas-api"
Jul 28 01:34:47.062065 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.062045    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:34:47.065400 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.065381    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-b70cb0d5-serving-cert\""
Jul 28 01:34:47.074541 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.074470    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"]
Jul 28 01:34:47.150161 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.150131    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp78c\" (UniqueName: \"kubernetes.io/projected/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-kube-api-access-sp78c\") pod \"maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb\" (UID: \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:34:47.150332 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.150170    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-maas-api-tls\") pod \"maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb\" (UID: \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:34:47.150332 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.150298    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-openshift-service-ca\") pod \"maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb\" (UID: \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:34:47.250990 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.250954    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp78c\" (UniqueName: \"kubernetes.io/projected/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-kube-api-access-sp78c\") pod \"maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb\" (UID: \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:34:47.250990 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.250993    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-maas-api-tls\") pod \"maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb\" (UID: \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:34:47.251215 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.251066    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-openshift-service-ca\") pod \"maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb\" (UID: \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:34:47.251215 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:34:47.251177    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-serving-cert: secret "maas-api-e2e-migrate-b70cb0d5-serving-cert" not found
Jul 28 01:34:47.251315 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:34:47.251249    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-maas-api-tls podName:b8b532a2-7931-4bcb-a5e0-8b2342a346ca nodeName:}" failed. No retries permitted until 2026-07-28 01:34:47.751233991 +0000 UTC m=+1838.796127941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-maas-api-tls") pod "maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb" (UID: "b8b532a2-7931-4bcb-a5e0-8b2342a346ca") : secret "maas-api-e2e-migrate-b70cb0d5-serving-cert" not found
Jul 28 01:34:47.251825 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.251804    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-openshift-service-ca\") pod \"maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb\" (UID: \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:34:47.264760 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.264736    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp78c\" (UniqueName: \"kubernetes.io/projected/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-kube-api-access-sp78c\") pod \"maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb\" (UID: \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:34:47.755237 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.755206    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-maas-api-tls\") pod \"maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb\" (UID: \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:34:47.758069 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.758040    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-maas-api-tls\") pod \"maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb\" (UID: \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:34:47.972605 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:47.972570    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:34:48.112189 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:48.112090    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"]
Jul 28 01:34:48.114588 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:34:48.114474    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8b532a2_7931_4bcb_a5e0_8b2342a346ca.slice/crio-a9a20a08840edcde00a09375399ac210262ee35916bb7a7c320c52d492bb3310 WatchSource:0}: Error finding container a9a20a08840edcde00a09375399ac210262ee35916bb7a7c320c52d492bb3310: Status 404 returned error can't find the container with id a9a20a08840edcde00a09375399ac210262ee35916bb7a7c320c52d492bb3310
Jul 28 01:34:48.466653 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:48.466628    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb" event={"ID":"b8b532a2-7931-4bcb-a5e0-8b2342a346ca","Type":"ContainerStarted","Data":"a9a20a08840edcde00a09375399ac210262ee35916bb7a7c320c52d492bb3310"}
Jul 28 01:34:49.472206 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:49.472174    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb" event={"ID":"b8b532a2-7931-4bcb-a5e0-8b2342a346ca","Type":"ContainerStarted","Data":"5cab53257b1f3049b1dc322a4783809061cd56e720654228c50929f8a838556f"}
Jul 28 01:34:49.472586 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:49.472226    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:34:49.492017 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:49.491972    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb" podStartSLOduration=2.159901354 podStartE2EDuration="2.49195988s" totalImagesPullingTime="332.058526ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:34:49.490373242 +0000 UTC m=+1840.535267210" watchObservedRunningTime="2026-07-28 01:34:49.49195988 +0000 UTC m=+1840.536853849"
Jul 28 01:34:55.484248 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:34:55.484217    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:35:18.451984 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:18.451949    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"]
Jul 28 01:35:18.452474 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:18.452187    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb" podUID="b8b532a2-7931-4bcb-a5e0-8b2342a346ca" containerName="maas-api" containerID="cri-o://5cab53257b1f3049b1dc322a4783809061cd56e720654228c50929f8a838556f" gracePeriod=30
Jul 28 01:35:18.605926 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:18.605882    2565 generic.go:356] "Generic (PLEG): container finished" podID="b8b532a2-7931-4bcb-a5e0-8b2342a346ca" containerID="5cab53257b1f3049b1dc322a4783809061cd56e720654228c50929f8a838556f" exitCode=0
Jul 28 01:35:18.606067 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:18.605949    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb" event={"ID":"b8b532a2-7931-4bcb-a5e0-8b2342a346ca","Type":"ContainerDied","Data":"5cab53257b1f3049b1dc322a4783809061cd56e720654228c50929f8a838556f"}
Jul 28 01:35:18.689698 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:18.689677    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:35:18.830784 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:18.830754    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sp78c\" (UniqueName: \"kubernetes.io/projected/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-kube-api-access-sp78c\") pod \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\" (UID: \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\") "
Jul 28 01:35:18.830970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:18.830800    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-maas-api-tls\") pod \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\" (UID: \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\") "
Jul 28 01:35:18.830970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:18.830861    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-openshift-service-ca\") pod \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\" (UID: \"b8b532a2-7931-4bcb-a5e0-8b2342a346ca\") "
Jul 28 01:35:18.831254 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:18.831230    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b8b532a2-7931-4bcb-a5e0-8b2342a346ca" (UID: "b8b532a2-7931-4bcb-a5e0-8b2342a346ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:35:18.832990 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:18.832968    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-kube-api-access-sp78c" (OuterVolumeSpecName: "kube-api-access-sp78c") pod "b8b532a2-7931-4bcb-a5e0-8b2342a346ca" (UID: "b8b532a2-7931-4bcb-a5e0-8b2342a346ca"). InnerVolumeSpecName "kube-api-access-sp78c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:35:18.833127 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:18.833115    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b8b532a2-7931-4bcb-a5e0-8b2342a346ca" (UID: "b8b532a2-7931-4bcb-a5e0-8b2342a346ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:35:18.932209 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:18.932183    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:35:18.932209 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:18.932205    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:35:18.932209 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:18.932214    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sp78c\" (UniqueName: \"kubernetes.io/projected/b8b532a2-7931-4bcb-a5e0-8b2342a346ca-kube-api-access-sp78c\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:35:19.612236 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:19.612202    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"
Jul 28 01:35:19.612236 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:19.612230    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb" event={"ID":"b8b532a2-7931-4bcb-a5e0-8b2342a346ca","Type":"ContainerDied","Data":"a9a20a08840edcde00a09375399ac210262ee35916bb7a7c320c52d492bb3310"}
Jul 28 01:35:19.612678 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:19.612290    2565 scope.go:117] "RemoveContainer" containerID="5cab53257b1f3049b1dc322a4783809061cd56e720654228c50929f8a838556f"
Jul 28 01:35:19.630016 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:19.629994    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"]
Jul 28 01:35:19.634151 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:19.634131    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-b70cb0d5-789b94d547-dnrnb"]
Jul 28 01:35:21.467420 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:21.467386    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8b532a2-7931-4bcb-a5e0-8b2342a346ca" path="/var/lib/kubelet/pods/b8b532a2-7931-4bcb-a5e0-8b2342a346ca/volumes"
Jul 28 01:35:22.200531 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.200496    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"]
Jul 28 01:35:22.201032 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.201018    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8b532a2-7931-4bcb-a5e0-8b2342a346ca" containerName="maas-api"
Jul 28 01:35:22.201102 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.201035    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8b532a2-7931-4bcb-a5e0-8b2342a346ca" containerName="maas-api"
Jul 28 01:35:22.201162 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.201150    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8b532a2-7931-4bcb-a5e0-8b2342a346ca" containerName="maas-api"
Jul 28 01:35:22.205896 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.205877    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:22.208738 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.208716    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0fcf74cb-serving-cert\""
Jul 28 01:35:22.216571 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.216548    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"]
Jul 28 01:35:22.365155 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.365123    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6896b077-5dcc-4588-bd17-ab83d0d39e7c-maas-api-tls\") pod \"maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x\" (UID: \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:22.365301 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.365172    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5sfw\" (UniqueName: \"kubernetes.io/projected/6896b077-5dcc-4588-bd17-ab83d0d39e7c-kube-api-access-n5sfw\") pod \"maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x\" (UID: \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:22.365301 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.365279    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6896b077-5dcc-4588-bd17-ab83d0d39e7c-openshift-service-ca\") pod \"maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x\" (UID: \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:22.465952 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.465872    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6896b077-5dcc-4588-bd17-ab83d0d39e7c-maas-api-tls\") pod \"maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x\" (UID: \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:22.465952 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.465929    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5sfw\" (UniqueName: \"kubernetes.io/projected/6896b077-5dcc-4588-bd17-ab83d0d39e7c-kube-api-access-n5sfw\") pod \"maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x\" (UID: \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:22.466170 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.466002    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6896b077-5dcc-4588-bd17-ab83d0d39e7c-openshift-service-ca\") pod \"maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x\" (UID: \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:22.466669 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.466646    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6896b077-5dcc-4588-bd17-ab83d0d39e7c-openshift-service-ca\") pod \"maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x\" (UID: \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:22.468582 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.468558    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6896b077-5dcc-4588-bd17-ab83d0d39e7c-maas-api-tls\") pod \"maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x\" (UID: \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:22.474703 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.474679    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5sfw\" (UniqueName: \"kubernetes.io/projected/6896b077-5dcc-4588-bd17-ab83d0d39e7c-kube-api-access-n5sfw\") pod \"maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x\" (UID: \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:22.517414 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.517388    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:22.655519 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:22.655385    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"]
Jul 28 01:35:22.657709 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:35:22.657678    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6896b077_5dcc_4588_bd17_ab83d0d39e7c.slice/crio-b8ee478433326082ce411785a9d913780e76671a23f68d37a868ee5acec83d19 WatchSource:0}: Error finding container b8ee478433326082ce411785a9d913780e76671a23f68d37a868ee5acec83d19: Status 404 returned error can't find the container with id b8ee478433326082ce411785a9d913780e76671a23f68d37a868ee5acec83d19
Jul 28 01:35:23.634973 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:23.634939    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x" event={"ID":"6896b077-5dcc-4588-bd17-ab83d0d39e7c","Type":"ContainerStarted","Data":"c096fe79756261c59d155f2f3514fb91ffeb0ff0e35e2525b2f104645151e684"}
Jul 28 01:35:23.635359 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:23.634995    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x" event={"ID":"6896b077-5dcc-4588-bd17-ab83d0d39e7c","Type":"ContainerStarted","Data":"b8ee478433326082ce411785a9d913780e76671a23f68d37a868ee5acec83d19"}
Jul 28 01:35:23.635359 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:23.635022    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:23.657187 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:23.657141    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x" podStartSLOduration=1.139922168 podStartE2EDuration="1.657126344s" totalImagesPullingTime="517.204176ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:35:23.655668078 +0000 UTC m=+1874.700562042" watchObservedRunningTime="2026-07-28 01:35:23.657126344 +0000 UTC m=+1874.702020313"
Jul 28 01:35:29.646177 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:29.646104    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:52.825444 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:52.825412    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"]
Jul 28 01:35:52.825995 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:52.825685    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x" podUID="6896b077-5dcc-4588-bd17-ab83d0d39e7c" containerName="maas-api" containerID="cri-o://c096fe79756261c59d155f2f3514fb91ffeb0ff0e35e2525b2f104645151e684" gracePeriod=30
Jul 28 01:35:53.072822 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.072798    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:53.149233 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.149155    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6896b077-5dcc-4588-bd17-ab83d0d39e7c-openshift-service-ca\") pod \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\" (UID: \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\") "
Jul 28 01:35:53.149233 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.149191    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5sfw\" (UniqueName: \"kubernetes.io/projected/6896b077-5dcc-4588-bd17-ab83d0d39e7c-kube-api-access-n5sfw\") pod \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\" (UID: \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\") "
Jul 28 01:35:53.149430 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.149236    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6896b077-5dcc-4588-bd17-ab83d0d39e7c-maas-api-tls\") pod \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\" (UID: \"6896b077-5dcc-4588-bd17-ab83d0d39e7c\") "
Jul 28 01:35:53.149548 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.149521    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6896b077-5dcc-4588-bd17-ab83d0d39e7c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6896b077-5dcc-4588-bd17-ab83d0d39e7c" (UID: "6896b077-5dcc-4588-bd17-ab83d0d39e7c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:35:53.151390 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.151367    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6896b077-5dcc-4588-bd17-ab83d0d39e7c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6896b077-5dcc-4588-bd17-ab83d0d39e7c" (UID: "6896b077-5dcc-4588-bd17-ab83d0d39e7c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:35:53.151627 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.151607    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6896b077-5dcc-4588-bd17-ab83d0d39e7c-kube-api-access-n5sfw" (OuterVolumeSpecName: "kube-api-access-n5sfw") pod "6896b077-5dcc-4588-bd17-ab83d0d39e7c" (UID: "6896b077-5dcc-4588-bd17-ab83d0d39e7c"). InnerVolumeSpecName "kube-api-access-n5sfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:35:53.250724 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.250689    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6896b077-5dcc-4588-bd17-ab83d0d39e7c-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:35:53.250724 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.250717    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5sfw\" (UniqueName: \"kubernetes.io/projected/6896b077-5dcc-4588-bd17-ab83d0d39e7c-kube-api-access-n5sfw\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:35:53.250724 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.250728    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6896b077-5dcc-4588-bd17-ab83d0d39e7c-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:35:53.777014 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.776915    2565 generic.go:356] "Generic (PLEG): container finished" podID="6896b077-5dcc-4588-bd17-ab83d0d39e7c" containerID="c096fe79756261c59d155f2f3514fb91ffeb0ff0e35e2525b2f104645151e684" exitCode=0
Jul 28 01:35:53.777014 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.776965    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"
Jul 28 01:35:53.777229 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.777007    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x" event={"ID":"6896b077-5dcc-4588-bd17-ab83d0d39e7c","Type":"ContainerDied","Data":"c096fe79756261c59d155f2f3514fb91ffeb0ff0e35e2525b2f104645151e684"}
Jul 28 01:35:53.777229 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.777062    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x" event={"ID":"6896b077-5dcc-4588-bd17-ab83d0d39e7c","Type":"ContainerDied","Data":"b8ee478433326082ce411785a9d913780e76671a23f68d37a868ee5acec83d19"}
Jul 28 01:35:53.777229 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.777111    2565 scope.go:117] "RemoveContainer" containerID="c096fe79756261c59d155f2f3514fb91ffeb0ff0e35e2525b2f104645151e684"
Jul 28 01:35:53.786213 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.786193    2565 scope.go:117] "RemoveContainer" containerID="c096fe79756261c59d155f2f3514fb91ffeb0ff0e35e2525b2f104645151e684"
Jul 28 01:35:53.786538 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:35:53.786515    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c096fe79756261c59d155f2f3514fb91ffeb0ff0e35e2525b2f104645151e684\": container with ID starting with c096fe79756261c59d155f2f3514fb91ffeb0ff0e35e2525b2f104645151e684 not found: ID does not exist" containerID="c096fe79756261c59d155f2f3514fb91ffeb0ff0e35e2525b2f104645151e684"
Jul 28 01:35:53.786607 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.786550    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c096fe79756261c59d155f2f3514fb91ffeb0ff0e35e2525b2f104645151e684"} err="failed to get container status \"c096fe79756261c59d155f2f3514fb91ffeb0ff0e35e2525b2f104645151e684\": rpc error: code = NotFound desc = could not find container \"c096fe79756261c59d155f2f3514fb91ffeb0ff0e35e2525b2f104645151e684\": container with ID starting with c096fe79756261c59d155f2f3514fb91ffeb0ff0e35e2525b2f104645151e684 not found: ID does not exist"
Jul 28 01:35:53.794579 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.794555    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"]
Jul 28 01:35:53.796168 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:53.796146    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0fcf74cb-6f58d5d9bd-4m56x"]
Jul 28 01:35:55.467164 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:55.467134    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6896b077-5dcc-4588-bd17-ab83d0d39e7c" path="/var/lib/kubelet/pods/6896b077-5dcc-4588-bd17-ab83d0d39e7c/volumes"
Jul 28 01:35:57.330354 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.330321    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"]
Jul 28 01:35:57.330894 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.330878    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6896b077-5dcc-4588-bd17-ab83d0d39e7c" containerName="maas-api"
Jul 28 01:35:57.330937 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.330897    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6896b077-5dcc-4588-bd17-ab83d0d39e7c" containerName="maas-api"
Jul 28 01:35:57.331013 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.331002    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6896b077-5dcc-4588-bd17-ab83d0d39e7c" containerName="maas-api"
Jul 28 01:35:57.335791 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.335774    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:35:57.338406 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.338385    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-30dd5a44-serving-cert\""
Jul 28 01:35:57.346173 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.346150    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"]
Jul 28 01:35:57.494666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.494632    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21355419-8b5e-45e8-8614-d2f032e3624e-openshift-service-ca\") pod \"maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z\" (UID: \"21355419-8b5e-45e8-8614-d2f032e3624e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:35:57.494844 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.494688    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqzqq\" (UniqueName: \"kubernetes.io/projected/21355419-8b5e-45e8-8614-d2f032e3624e-kube-api-access-dqzqq\") pod \"maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z\" (UID: \"21355419-8b5e-45e8-8614-d2f032e3624e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:35:57.494844 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.494742    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21355419-8b5e-45e8-8614-d2f032e3624e-maas-api-tls\") pod \"maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z\" (UID: \"21355419-8b5e-45e8-8614-d2f032e3624e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:35:57.595655 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.595566    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21355419-8b5e-45e8-8614-d2f032e3624e-openshift-service-ca\") pod \"maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z\" (UID: \"21355419-8b5e-45e8-8614-d2f032e3624e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:35:57.595655 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.595607    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqzqq\" (UniqueName: \"kubernetes.io/projected/21355419-8b5e-45e8-8614-d2f032e3624e-kube-api-access-dqzqq\") pod \"maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z\" (UID: \"21355419-8b5e-45e8-8614-d2f032e3624e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:35:57.595827 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.595659    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21355419-8b5e-45e8-8614-d2f032e3624e-maas-api-tls\") pod \"maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z\" (UID: \"21355419-8b5e-45e8-8614-d2f032e3624e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:35:57.596271 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.596225    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21355419-8b5e-45e8-8614-d2f032e3624e-openshift-service-ca\") pod \"maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z\" (UID: \"21355419-8b5e-45e8-8614-d2f032e3624e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:35:57.598128 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.598105    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21355419-8b5e-45e8-8614-d2f032e3624e-maas-api-tls\") pod \"maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z\" (UID: \"21355419-8b5e-45e8-8614-d2f032e3624e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:35:57.604203 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.604182    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqzqq\" (UniqueName: \"kubernetes.io/projected/21355419-8b5e-45e8-8614-d2f032e3624e-kube-api-access-dqzqq\") pod \"maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z\" (UID: \"21355419-8b5e-45e8-8614-d2f032e3624e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:35:57.647925 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.647893    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:35:57.780745 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.780716    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"]
Jul 28 01:35:57.782735 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:35:57.782706    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21355419_8b5e_45e8_8614_d2f032e3624e.slice/crio-070b486f4fdaabbd389acd6ed3a260f310c5b43d24b2532b94f265209f2dbb22 WatchSource:0}: Error finding container 070b486f4fdaabbd389acd6ed3a260f310c5b43d24b2532b94f265209f2dbb22: Status 404 returned error can't find the container with id 070b486f4fdaabbd389acd6ed3a260f310c5b43d24b2532b94f265209f2dbb22
Jul 28 01:35:57.797332 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:57.797304    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z" event={"ID":"21355419-8b5e-45e8-8614-d2f032e3624e","Type":"ContainerStarted","Data":"070b486f4fdaabbd389acd6ed3a260f310c5b43d24b2532b94f265209f2dbb22"}
Jul 28 01:35:58.802909 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:58.802866    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z" event={"ID":"21355419-8b5e-45e8-8614-d2f032e3624e","Type":"ContainerStarted","Data":"3c3a33415beac752c0305b8d47b717ef9bb01be088ddfd6e5cda6c301402a24c"}
Jul 28 01:35:58.803390 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:58.802989    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:35:58.822250 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:35:58.822198    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z" podStartSLOduration=1.397922986 podStartE2EDuration="1.822182345s" totalImagesPullingTime="424.259359ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:35:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:35:58.820336275 +0000 UTC m=+1909.865230241" watchObservedRunningTime="2026-07-28 01:35:58.822182345 +0000 UTC m=+1909.867076311"
Jul 28 01:36:04.812568 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:04.812543    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:36:27.865841 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:27.865810    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"]
Jul 28 01:36:27.866383 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:27.866066    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z" podUID="21355419-8b5e-45e8-8614-d2f032e3624e" containerName="maas-api" containerID="cri-o://3c3a33415beac752c0305b8d47b717ef9bb01be088ddfd6e5cda6c301402a24c" gracePeriod=30
Jul 28 01:36:28.110315 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.110288    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:36:28.167129 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.167065    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqzqq\" (UniqueName: \"kubernetes.io/projected/21355419-8b5e-45e8-8614-d2f032e3624e-kube-api-access-dqzqq\") pod \"21355419-8b5e-45e8-8614-d2f032e3624e\" (UID: \"21355419-8b5e-45e8-8614-d2f032e3624e\") "
Jul 28 01:36:28.167129 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.167110    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21355419-8b5e-45e8-8614-d2f032e3624e-maas-api-tls\") pod \"21355419-8b5e-45e8-8614-d2f032e3624e\" (UID: \"21355419-8b5e-45e8-8614-d2f032e3624e\") "
Jul 28 01:36:28.167280 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.167147    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21355419-8b5e-45e8-8614-d2f032e3624e-openshift-service-ca\") pod \"21355419-8b5e-45e8-8614-d2f032e3624e\" (UID: \"21355419-8b5e-45e8-8614-d2f032e3624e\") "
Jul 28 01:36:28.167593 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.167567    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21355419-8b5e-45e8-8614-d2f032e3624e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "21355419-8b5e-45e8-8614-d2f032e3624e" (UID: "21355419-8b5e-45e8-8614-d2f032e3624e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:36:28.169411 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.169386    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21355419-8b5e-45e8-8614-d2f032e3624e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "21355419-8b5e-45e8-8614-d2f032e3624e" (UID: "21355419-8b5e-45e8-8614-d2f032e3624e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:36:28.169539 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.169387    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21355419-8b5e-45e8-8614-d2f032e3624e-kube-api-access-dqzqq" (OuterVolumeSpecName: "kube-api-access-dqzqq") pod "21355419-8b5e-45e8-8614-d2f032e3624e" (UID: "21355419-8b5e-45e8-8614-d2f032e3624e"). InnerVolumeSpecName "kube-api-access-dqzqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:36:28.268187 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.268158    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqzqq\" (UniqueName: \"kubernetes.io/projected/21355419-8b5e-45e8-8614-d2f032e3624e-kube-api-access-dqzqq\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:36:28.268187 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.268182    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21355419-8b5e-45e8-8614-d2f032e3624e-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:36:28.268187 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.268191    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21355419-8b5e-45e8-8614-d2f032e3624e-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:36:28.952943 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.952908    2565 generic.go:356] "Generic (PLEG): container finished" podID="21355419-8b5e-45e8-8614-d2f032e3624e" containerID="3c3a33415beac752c0305b8d47b717ef9bb01be088ddfd6e5cda6c301402a24c" exitCode=0
Jul 28 01:36:28.953384 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.952956    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"
Jul 28 01:36:28.953384 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.952962    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z" event={"ID":"21355419-8b5e-45e8-8614-d2f032e3624e","Type":"ContainerDied","Data":"3c3a33415beac752c0305b8d47b717ef9bb01be088ddfd6e5cda6c301402a24c"}
Jul 28 01:36:28.953384 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.953016    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z" event={"ID":"21355419-8b5e-45e8-8614-d2f032e3624e","Type":"ContainerDied","Data":"070b486f4fdaabbd389acd6ed3a260f310c5b43d24b2532b94f265209f2dbb22"}
Jul 28 01:36:28.953384 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.953048    2565 scope.go:117] "RemoveContainer" containerID="3c3a33415beac752c0305b8d47b717ef9bb01be088ddfd6e5cda6c301402a24c"
Jul 28 01:36:28.962032 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.962010    2565 scope.go:117] "RemoveContainer" containerID="3c3a33415beac752c0305b8d47b717ef9bb01be088ddfd6e5cda6c301402a24c"
Jul 28 01:36:28.962286 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:36:28.962268    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c3a33415beac752c0305b8d47b717ef9bb01be088ddfd6e5cda6c301402a24c\": container with ID starting with 3c3a33415beac752c0305b8d47b717ef9bb01be088ddfd6e5cda6c301402a24c not found: ID does not exist" containerID="3c3a33415beac752c0305b8d47b717ef9bb01be088ddfd6e5cda6c301402a24c"
Jul 28 01:36:28.962350 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:28.962293    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c3a33415beac752c0305b8d47b717ef9bb01be088ddfd6e5cda6c301402a24c"} err="failed to get container status \"3c3a33415beac752c0305b8d47b717ef9bb01be088ddfd6e5cda6c301402a24c\": rpc error: code = NotFound desc = could not find container \"3c3a33415beac752c0305b8d47b717ef9bb01be088ddfd6e5cda6c301402a24c\": container with ID starting with 3c3a33415beac752c0305b8d47b717ef9bb01be088ddfd6e5cda6c301402a24c not found: ID does not exist"
Jul 28 01:36:29.036853 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:29.036826    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"]
Jul 28 01:36:29.050582 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:29.050550    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-30dd5a44-67db9d4df5-fcv9z"]
Jul 28 01:36:29.466547 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:36:29.466515    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21355419-8b5e-45e8-8614-d2f032e3624e" path="/var/lib/kubelet/pods/21355419-8b5e-45e8-8614-d2f032e3624e/volumes"
Jul 28 01:38:25.929092 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:25.929059    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"]
Jul 28 01:38:25.929690 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:25.929409    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21355419-8b5e-45e8-8614-d2f032e3624e" containerName="maas-api"
Jul 28 01:38:25.929690 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:25.929419    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="21355419-8b5e-45e8-8614-d2f032e3624e" containerName="maas-api"
Jul 28 01:38:25.929690 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:25.929514    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="21355419-8b5e-45e8-8614-d2f032e3624e" containerName="maas-api"
Jul 28 01:38:25.932587 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:25.932571    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:38:25.936341 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:25.936322    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6046872f-serving-cert\""
Jul 28 01:38:25.945986 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:25.945964    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"]
Jul 28 01:38:25.954790 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:25.954768    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-maas-api-tls\") pod \"maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2\" (UID: \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:38:25.954872 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:25.954838    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn4cp\" (UniqueName: \"kubernetes.io/projected/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-kube-api-access-dn4cp\") pod \"maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2\" (UID: \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:38:25.954872 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:25.954868    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-openshift-service-ca\") pod \"maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2\" (UID: \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:38:26.056152 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:26.056120    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn4cp\" (UniqueName: \"kubernetes.io/projected/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-kube-api-access-dn4cp\") pod \"maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2\" (UID: \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:38:26.056328 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:26.056161    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-openshift-service-ca\") pod \"maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2\" (UID: \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:38:26.056328 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:26.056216    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-maas-api-tls\") pod \"maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2\" (UID: \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:38:26.056989 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:26.056952    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-openshift-service-ca\") pod \"maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2\" (UID: \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:38:26.058935 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:26.058915    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-maas-api-tls\") pod \"maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2\" (UID: \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:38:26.066285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:26.066265    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn4cp\" (UniqueName: \"kubernetes.io/projected/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-kube-api-access-dn4cp\") pod \"maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2\" (UID: \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:38:26.242994 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:26.242915    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:38:26.591983 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:26.591958    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"]
Jul 28 01:38:26.593828 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:38:26.593802    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c8a7be2_e84f_4b7d_82d8_6740bde2e8cd.slice/crio-6544ecc39d5f97374368d76141998ff81d948e18726226f688fa8035895e0d71 WatchSource:0}: Error finding container 6544ecc39d5f97374368d76141998ff81d948e18726226f688fa8035895e0d71: Status 404 returned error can't find the container with id 6544ecc39d5f97374368d76141998ff81d948e18726226f688fa8035895e0d71
Jul 28 01:38:27.515663 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:27.515622    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2" event={"ID":"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd","Type":"ContainerStarted","Data":"b1858861603b14657189ef3a3dcf82251ae9f87d243a7ae27b32fb098ab3dc86"}
Jul 28 01:38:27.516098 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:27.515681    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2" event={"ID":"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd","Type":"ContainerStarted","Data":"6544ecc39d5f97374368d76141998ff81d948e18726226f688fa8035895e0d71"}
Jul 28 01:38:27.516098 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:27.515708    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:38:27.535330 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:27.535280    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2" podStartSLOduration=2.126132574 podStartE2EDuration="2.535266151s" totalImagesPullingTime="409.133577ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:38:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:38:27.534671995 +0000 UTC m=+2058.579565959" watchObservedRunningTime="2026-07-28 01:38:27.535266151 +0000 UTC m=+2058.580160258"
Jul 28 01:38:33.528081 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:38:33.528054    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:39:08.394602 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.394569    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"]
Jul 28 01:39:08.395201 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.394827    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2" podUID="9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd" containerName="maas-api" containerID="cri-o://b1858861603b14657189ef3a3dcf82251ae9f87d243a7ae27b32fb098ab3dc86" gracePeriod=30
Jul 28 01:39:08.522372 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.522339    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2" podUID="9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.39:8443/health\": dial tcp 10.132.0.39:8443: connect: connection refused"
Jul 28 01:39:08.641382 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.641358    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:39:08.661099 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.661026    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-maas-api-tls\") pod \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\" (UID: \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\") "
Jul 28 01:39:08.661224 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.661119    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn4cp\" (UniqueName: \"kubernetes.io/projected/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-kube-api-access-dn4cp\") pod \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\" (UID: \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\") "
Jul 28 01:39:08.661297 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.661248    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-openshift-service-ca\") pod \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\" (UID: \"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd\") "
Jul 28 01:39:08.661864 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.661788    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd" (UID: "9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:39:08.663658 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.663620    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-kube-api-access-dn4cp" (OuterVolumeSpecName: "kube-api-access-dn4cp") pod "9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd" (UID: "9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd"). InnerVolumeSpecName "kube-api-access-dn4cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:39:08.663868 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.663840    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd" (UID: "9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:39:08.714317 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.714282    2565 generic.go:356] "Generic (PLEG): container finished" podID="9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd" containerID="b1858861603b14657189ef3a3dcf82251ae9f87d243a7ae27b32fb098ab3dc86" exitCode=0
Jul 28 01:39:08.714534 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.714324    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"
Jul 28 01:39:08.714534 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.714343    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2" event={"ID":"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd","Type":"ContainerDied","Data":"b1858861603b14657189ef3a3dcf82251ae9f87d243a7ae27b32fb098ab3dc86"}
Jul 28 01:39:08.714534 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.714398    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2" event={"ID":"9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd","Type":"ContainerDied","Data":"6544ecc39d5f97374368d76141998ff81d948e18726226f688fa8035895e0d71"}
Jul 28 01:39:08.714534 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.714430    2565 scope.go:117] "RemoveContainer" containerID="b1858861603b14657189ef3a3dcf82251ae9f87d243a7ae27b32fb098ab3dc86"
Jul 28 01:39:08.724029 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.724012    2565 scope.go:117] "RemoveContainer" containerID="b1858861603b14657189ef3a3dcf82251ae9f87d243a7ae27b32fb098ab3dc86"
Jul 28 01:39:08.724275 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:39:08.724252    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1858861603b14657189ef3a3dcf82251ae9f87d243a7ae27b32fb098ab3dc86\": container with ID starting with b1858861603b14657189ef3a3dcf82251ae9f87d243a7ae27b32fb098ab3dc86 not found: ID does not exist" containerID="b1858861603b14657189ef3a3dcf82251ae9f87d243a7ae27b32fb098ab3dc86"
Jul 28 01:39:08.724338 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.724287    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1858861603b14657189ef3a3dcf82251ae9f87d243a7ae27b32fb098ab3dc86"} err="failed to get container status \"b1858861603b14657189ef3a3dcf82251ae9f87d243a7ae27b32fb098ab3dc86\": rpc error: code = NotFound desc = could not find container \"b1858861603b14657189ef3a3dcf82251ae9f87d243a7ae27b32fb098ab3dc86\": container with ID starting with b1858861603b14657189ef3a3dcf82251ae9f87d243a7ae27b32fb098ab3dc86 not found: ID does not exist"
Jul 28 01:39:08.737060 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.737039    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"]
Jul 28 01:39:08.741303 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.741278    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6046872f-c9d94fb49-rkjn2"]
Jul 28 01:39:08.762779 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.762760    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:39:08.762860 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.762781    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:39:08.762860 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:08.762791    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn4cp\" (UniqueName: \"kubernetes.io/projected/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd-kube-api-access-dn4cp\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:39:09.468371 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:09.468338    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd" path="/var/lib/kubelet/pods/9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd/volumes"
Jul 28 01:39:10.193712 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:10.193609    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:39:10.207762 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:10.194645    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:39:10.207762 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:10.200101    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:39:10.207762 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:10.201017    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:39:10.207762 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:10.201157    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:39:10.207762 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:10.207370    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:39:38.219959 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.219924    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"]
Jul 28 01:39:38.220443 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.220424    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd" containerName="maas-api"
Jul 28 01:39:38.220508 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.220448    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd" containerName="maas-api"
Jul 28 01:39:38.220585 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.220573    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c8a7be2-e84f-4b7d-82d8-6740bde2e8cd" containerName="maas-api"
Jul 28 01:39:38.223855 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.223838    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:39:38.226508 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.226461    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f0341a4c-serving-cert\""
Jul 28 01:39:38.235498 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.235468    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"]
Jul 28 01:39:38.327567 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.327539    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjkx6\" (UniqueName: \"kubernetes.io/projected/c81907ea-5b5d-4c28-ab45-7912564ed055-kube-api-access-vjkx6\") pod \"maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g\" (UID: \"c81907ea-5b5d-4c28-ab45-7912564ed055\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:39:38.327700 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.327580    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c81907ea-5b5d-4c28-ab45-7912564ed055-openshift-service-ca\") pod \"maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g\" (UID: \"c81907ea-5b5d-4c28-ab45-7912564ed055\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:39:38.327700 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.327676    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c81907ea-5b5d-4c28-ab45-7912564ed055-maas-api-tls\") pod \"maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g\" (UID: \"c81907ea-5b5d-4c28-ab45-7912564ed055\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:39:38.428771 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.428737    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c81907ea-5b5d-4c28-ab45-7912564ed055-maas-api-tls\") pod \"maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g\" (UID: \"c81907ea-5b5d-4c28-ab45-7912564ed055\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:39:38.428930 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.428814    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjkx6\" (UniqueName: \"kubernetes.io/projected/c81907ea-5b5d-4c28-ab45-7912564ed055-kube-api-access-vjkx6\") pod \"maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g\" (UID: \"c81907ea-5b5d-4c28-ab45-7912564ed055\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:39:38.428930 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.428844    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c81907ea-5b5d-4c28-ab45-7912564ed055-openshift-service-ca\") pod \"maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g\" (UID: \"c81907ea-5b5d-4c28-ab45-7912564ed055\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:39:38.429433 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.429416    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c81907ea-5b5d-4c28-ab45-7912564ed055-openshift-service-ca\") pod \"maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g\" (UID: \"c81907ea-5b5d-4c28-ab45-7912564ed055\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:39:38.431245 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.431213    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c81907ea-5b5d-4c28-ab45-7912564ed055-maas-api-tls\") pod \"maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g\" (UID: \"c81907ea-5b5d-4c28-ab45-7912564ed055\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:39:38.437382 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.437360    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjkx6\" (UniqueName: \"kubernetes.io/projected/c81907ea-5b5d-4c28-ab45-7912564ed055-kube-api-access-vjkx6\") pod \"maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g\" (UID: \"c81907ea-5b5d-4c28-ab45-7912564ed055\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:39:38.534187 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.534124    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:39:38.670397 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.670374    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"]
Jul 28 01:39:38.672902 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:39:38.672854    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc81907ea_5b5d_4c28_ab45_7912564ed055.slice/crio-4c7e6fa407357c396d0b09e0f1482730fc8a793a5e83dd84c92a09efea78d58b WatchSource:0}: Error finding container 4c7e6fa407357c396d0b09e0f1482730fc8a793a5e83dd84c92a09efea78d58b: Status 404 returned error can't find the container with id 4c7e6fa407357c396d0b09e0f1482730fc8a793a5e83dd84c92a09efea78d58b
Jul 28 01:39:38.674525 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.674472    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:39:38.853743 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:38.853711    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g" event={"ID":"c81907ea-5b5d-4c28-ab45-7912564ed055","Type":"ContainerStarted","Data":"4c7e6fa407357c396d0b09e0f1482730fc8a793a5e83dd84c92a09efea78d58b"}
Jul 28 01:39:39.859494 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:39.859459    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g" event={"ID":"c81907ea-5b5d-4c28-ab45-7912564ed055","Type":"ContainerStarted","Data":"a22ef576fe1e2241e7fa2899b9830337fd4ef4f92a5db0c0ef74cf6d1a868b56"}
Jul 28 01:39:39.859920 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:39.859536    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:39:39.877425 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:39.877372    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g" podStartSLOduration=1.510836037 podStartE2EDuration="1.877356503s" totalImagesPullingTime="366.520466ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:39:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:39:39.876141383 +0000 UTC m=+2130.921035353" watchObservedRunningTime="2026-07-28 01:39:39.877356503 +0000 UTC m=+2130.922250471"
Jul 28 01:39:45.870776 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:39:45.870736    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:40:08.117977 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.117900    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"]
Jul 28 01:40:08.121167 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.121150    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.125207 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.125182    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f0341a4c\"/\"e2e-lifecycle-model-f0341a4c-kserve-self-signed-certs\""
Jul 28 01:40:08.125207 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.125204    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f0341a4c\"/\"default-dockercfg-wb7wz\""
Jul 28 01:40:08.125402 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.125240    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f0341a4c\"/\"kube-root-ca.crt\""
Jul 28 01:40:08.125402 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.125292    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f0341a4c\"/\"openshift-service-ca.crt\""
Jul 28 01:40:08.131800 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.131781    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"]
Jul 28 01:40:08.192931 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.192906    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-tmp-dir\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.193051 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.192953    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-home\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.193091 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.193044    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-model-cache\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.193129 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.193095    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-dshm\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.193163 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.193128    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0c2aa7f-403d-4f1e-8643-db36a47df246-tls-certs\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.193163 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.193149    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x96zv\" (UniqueName: \"kubernetes.io/projected/e0c2aa7f-403d-4f1e-8643-db36a47df246-kube-api-access-x96zv\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.293677 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.293649    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-tmp-dir\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.293849 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.293694    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-home\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.293849 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.293732    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-model-cache\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.293849 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.293760    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-dshm\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.293849 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.293776    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0c2aa7f-403d-4f1e-8643-db36a47df246-tls-certs\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.293849 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.293792    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x96zv\" (UniqueName: \"kubernetes.io/projected/e0c2aa7f-403d-4f1e-8643-db36a47df246-kube-api-access-x96zv\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.294158 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.294092    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-tmp-dir\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.294158 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.294113    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-home\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.294266 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.294171    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-model-cache\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.296228 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.296199    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-dshm\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.296397 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.296377    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0c2aa7f-403d-4f1e-8643-db36a47df246-tls-certs\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.302951 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.302933    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x96zv\" (UniqueName: \"kubernetes.io/projected/e0c2aa7f-403d-4f1e-8643-db36a47df246-kube-api-access-x96zv\") pod \"e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") " pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.431279 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.431214    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:08.563621 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:08.563594    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"]
Jul 28 01:40:08.565856 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:40:08.565828    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0c2aa7f_403d_4f1e_8643_db36a47df246.slice/crio-cc9291c5ab155167d46ebe5d4e448b0f19ac4e6f47363cd273e888068f6114be WatchSource:0}: Error finding container cc9291c5ab155167d46ebe5d4e448b0f19ac4e6f47363cd273e888068f6114be: Status 404 returned error can't find the container with id cc9291c5ab155167d46ebe5d4e448b0f19ac4e6f47363cd273e888068f6114be
Jul 28 01:40:09.001888 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:09.001848    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247" event={"ID":"e0c2aa7f-403d-4f1e-8643-db36a47df246","Type":"ContainerStarted","Data":"4537a4a39ee74b74a2a4be93eba97813de56f3d10747c7b0b1cca964c1593477"}
Jul 28 01:40:09.001888 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:09.001895    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247" event={"ID":"e0c2aa7f-403d-4f1e-8643-db36a47df246","Type":"ContainerStarted","Data":"cc9291c5ab155167d46ebe5d4e448b0f19ac4e6f47363cd273e888068f6114be"}
Jul 28 01:40:09.021414 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:09.021368    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247" podStartSLOduration=1.021354602 podStartE2EDuration="1.021354602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:40:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:40:09.018789233 +0000 UTC m=+2160.063683202" watchObservedRunningTime="2026-07-28 01:40:09.021354602 +0000 UTC m=+2160.066248569"
Jul 28 01:40:18.431975 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:18.431940    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:18.432427 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:18.432011    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:18.444386 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:18.444367    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:19.061037 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:19.061008    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:40:48.042344 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:48.042311    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"]
Jul 28 01:40:48.042798 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:48.042589    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g" podUID="c81907ea-5b5d-4c28-ab45-7912564ed055" containerName="maas-api" containerID="cri-o://a22ef576fe1e2241e7fa2899b9830337fd4ef4f92a5db0c0ef74cf6d1a868b56" gracePeriod=30
Jul 28 01:40:48.187251 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:48.187215    2565 generic.go:356] "Generic (PLEG): container finished" podID="c81907ea-5b5d-4c28-ab45-7912564ed055" containerID="a22ef576fe1e2241e7fa2899b9830337fd4ef4f92a5db0c0ef74cf6d1a868b56" exitCode=0
Jul 28 01:40:48.187410 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:48.187287    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g" event={"ID":"c81907ea-5b5d-4c28-ab45-7912564ed055","Type":"ContainerDied","Data":"a22ef576fe1e2241e7fa2899b9830337fd4ef4f92a5db0c0ef74cf6d1a868b56"}
Jul 28 01:40:48.310814 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:48.310788    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:40:48.350790 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:48.350765    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c81907ea-5b5d-4c28-ab45-7912564ed055-openshift-service-ca\") pod \"c81907ea-5b5d-4c28-ab45-7912564ed055\" (UID: \"c81907ea-5b5d-4c28-ab45-7912564ed055\") "
Jul 28 01:40:48.350962 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:48.350808    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c81907ea-5b5d-4c28-ab45-7912564ed055-maas-api-tls\") pod \"c81907ea-5b5d-4c28-ab45-7912564ed055\" (UID: \"c81907ea-5b5d-4c28-ab45-7912564ed055\") "
Jul 28 01:40:48.350962 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:48.350834    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjkx6\" (UniqueName: \"kubernetes.io/projected/c81907ea-5b5d-4c28-ab45-7912564ed055-kube-api-access-vjkx6\") pod \"c81907ea-5b5d-4c28-ab45-7912564ed055\" (UID: \"c81907ea-5b5d-4c28-ab45-7912564ed055\") "
Jul 28 01:40:48.351204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:48.351172    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c81907ea-5b5d-4c28-ab45-7912564ed055-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c81907ea-5b5d-4c28-ab45-7912564ed055" (UID: "c81907ea-5b5d-4c28-ab45-7912564ed055"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:40:48.353075 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:48.353047    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c81907ea-5b5d-4c28-ab45-7912564ed055-kube-api-access-vjkx6" (OuterVolumeSpecName: "kube-api-access-vjkx6") pod "c81907ea-5b5d-4c28-ab45-7912564ed055" (UID: "c81907ea-5b5d-4c28-ab45-7912564ed055"). InnerVolumeSpecName "kube-api-access-vjkx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:40:48.353175 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:48.353083    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c81907ea-5b5d-4c28-ab45-7912564ed055-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c81907ea-5b5d-4c28-ab45-7912564ed055" (UID: "c81907ea-5b5d-4c28-ab45-7912564ed055"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:40:48.452308 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:48.452277    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c81907ea-5b5d-4c28-ab45-7912564ed055-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:40:48.452308 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:48.452305    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c81907ea-5b5d-4c28-ab45-7912564ed055-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:40:48.452308 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:48.452318    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjkx6\" (UniqueName: \"kubernetes.io/projected/c81907ea-5b5d-4c28-ab45-7912564ed055-kube-api-access-vjkx6\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:40:49.194329 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:49.194295    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"
Jul 28 01:40:49.194957 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:49.194327    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g" event={"ID":"c81907ea-5b5d-4c28-ab45-7912564ed055","Type":"ContainerDied","Data":"4c7e6fa407357c396d0b09e0f1482730fc8a793a5e83dd84c92a09efea78d58b"}
Jul 28 01:40:49.194957 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:49.194385    2565 scope.go:117] "RemoveContainer" containerID="a22ef576fe1e2241e7fa2899b9830337fd4ef4f92a5db0c0ef74cf6d1a868b56"
Jul 28 01:40:49.219954 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:49.219918    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"]
Jul 28 01:40:49.223902 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:49.223877    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f0341a4c-65ff57f7fd-4vd8g"]
Jul 28 01:40:49.467986 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:49.467908    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c81907ea-5b5d-4c28-ab45-7912564ed055" path="/var/lib/kubelet/pods/c81907ea-5b5d-4c28-ab45-7912564ed055/volumes"
Jul 28 01:40:56.635680 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:40:56.635639    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"]
Jul 28 01:41:11.655616 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:11.655553    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247" podUID="e0c2aa7f-403d-4f1e-8643-db36a47df246" containerName="main" containerID="cri-o://4537a4a39ee74b74a2a4be93eba97813de56f3d10747c7b0b1cca964c1593477" gracePeriod=45
Jul 28 01:41:11.911205 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:11.911146    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:41:12.080748 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.080711    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-tmp-dir\") pod \"e0c2aa7f-403d-4f1e-8643-db36a47df246\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") "
Jul 28 01:41:12.080921 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.080785    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0c2aa7f-403d-4f1e-8643-db36a47df246-tls-certs\") pod \"e0c2aa7f-403d-4f1e-8643-db36a47df246\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") "
Jul 28 01:41:12.080921 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.080828    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-home\") pod \"e0c2aa7f-403d-4f1e-8643-db36a47df246\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") "
Jul 28 01:41:12.080921 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.080848    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-dshm\") pod \"e0c2aa7f-403d-4f1e-8643-db36a47df246\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") "
Jul 28 01:41:12.080921 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.080870    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x96zv\" (UniqueName: \"kubernetes.io/projected/e0c2aa7f-403d-4f1e-8643-db36a47df246-kube-api-access-x96zv\") pod \"e0c2aa7f-403d-4f1e-8643-db36a47df246\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") "
Jul 28 01:41:12.081138 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.080926    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-model-cache\") pod \"e0c2aa7f-403d-4f1e-8643-db36a47df246\" (UID: \"e0c2aa7f-403d-4f1e-8643-db36a47df246\") "
Jul 28 01:41:12.081138 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.081028    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e0c2aa7f-403d-4f1e-8643-db36a47df246" (UID: "e0c2aa7f-403d-4f1e-8643-db36a47df246"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:41:12.081229 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.081132    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-tmp-dir\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:41:12.081229 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.081133    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-home" (OuterVolumeSpecName: "home") pod "e0c2aa7f-403d-4f1e-8643-db36a47df246" (UID: "e0c2aa7f-403d-4f1e-8643-db36a47df246"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:41:12.081302 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.081281    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-model-cache" (OuterVolumeSpecName: "model-cache") pod "e0c2aa7f-403d-4f1e-8643-db36a47df246" (UID: "e0c2aa7f-403d-4f1e-8643-db36a47df246"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:41:12.083160 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.083139    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0c2aa7f-403d-4f1e-8643-db36a47df246-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e0c2aa7f-403d-4f1e-8643-db36a47df246" (UID: "e0c2aa7f-403d-4f1e-8643-db36a47df246"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:41:12.083228 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.083176    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0c2aa7f-403d-4f1e-8643-db36a47df246-kube-api-access-x96zv" (OuterVolumeSpecName: "kube-api-access-x96zv") pod "e0c2aa7f-403d-4f1e-8643-db36a47df246" (UID: "e0c2aa7f-403d-4f1e-8643-db36a47df246"). InnerVolumeSpecName "kube-api-access-x96zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:41:12.083228 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.083211    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-dshm" (OuterVolumeSpecName: "dshm") pod "e0c2aa7f-403d-4f1e-8643-db36a47df246" (UID: "e0c2aa7f-403d-4f1e-8643-db36a47df246"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:41:12.182168 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.182094    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0c2aa7f-403d-4f1e-8643-db36a47df246-tls-certs\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:41:12.182168 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.182118    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-home\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:41:12.182168 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.182126    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-dshm\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:41:12.182168 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.182137    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x96zv\" (UniqueName: \"kubernetes.io/projected/e0c2aa7f-403d-4f1e-8643-db36a47df246-kube-api-access-x96zv\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:41:12.182168 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.182147    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0c2aa7f-403d-4f1e-8643-db36a47df246-model-cache\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:41:12.305424 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.305388    2565 generic.go:356] "Generic (PLEG): container finished" podID="e0c2aa7f-403d-4f1e-8643-db36a47df246" containerID="4537a4a39ee74b74a2a4be93eba97813de56f3d10747c7b0b1cca964c1593477" exitCode=0
Jul 28 01:41:12.305624 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.305430    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"
Jul 28 01:41:12.305624 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.305467    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247" event={"ID":"e0c2aa7f-403d-4f1e-8643-db36a47df246","Type":"ContainerDied","Data":"4537a4a39ee74b74a2a4be93eba97813de56f3d10747c7b0b1cca964c1593477"}
Jul 28 01:41:12.305624 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.305523    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247" event={"ID":"e0c2aa7f-403d-4f1e-8643-db36a47df246","Type":"ContainerDied","Data":"cc9291c5ab155167d46ebe5d4e448b0f19ac4e6f47363cd273e888068f6114be"}
Jul 28 01:41:12.305624 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.305544    2565 scope.go:117] "RemoveContainer" containerID="4537a4a39ee74b74a2a4be93eba97813de56f3d10747c7b0b1cca964c1593477"
Jul 28 01:41:12.314607 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.314590    2565 scope.go:117] "RemoveContainer" containerID="4537a4a39ee74b74a2a4be93eba97813de56f3d10747c7b0b1cca964c1593477"
Jul 28 01:41:12.314850 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:41:12.314833    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4537a4a39ee74b74a2a4be93eba97813de56f3d10747c7b0b1cca964c1593477\": container with ID starting with 4537a4a39ee74b74a2a4be93eba97813de56f3d10747c7b0b1cca964c1593477 not found: ID does not exist" containerID="4537a4a39ee74b74a2a4be93eba97813de56f3d10747c7b0b1cca964c1593477"
Jul 28 01:41:12.314898 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.314861    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4537a4a39ee74b74a2a4be93eba97813de56f3d10747c7b0b1cca964c1593477"} err="failed to get container status \"4537a4a39ee74b74a2a4be93eba97813de56f3d10747c7b0b1cca964c1593477\": rpc error: code = NotFound desc = could not find container \"4537a4a39ee74b74a2a4be93eba97813de56f3d10747c7b0b1cca964c1593477\": container with ID starting with 4537a4a39ee74b74a2a4be93eba97813de56f3d10747c7b0b1cca964c1593477 not found: ID does not exist"
Jul 28 01:41:12.327149 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.327125    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"]
Jul 28 01:41:12.330969 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:12.330945    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-f0341a4c/e2e-lifecycle-model-f0341a4c-kserve-c69875f5-vn247"]
Jul 28 01:41:13.467516 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:41:13.467471    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0c2aa7f-403d-4f1e-8643-db36a47df246" path="/var/lib/kubelet/pods/e0c2aa7f-403d-4f1e-8643-db36a47df246/volumes"
Jul 28 01:42:38.430700 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.430662    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"]
Jul 28 01:42:38.431471 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.431448    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c81907ea-5b5d-4c28-ab45-7912564ed055" containerName="maas-api"
Jul 28 01:42:38.431722 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.431695    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c81907ea-5b5d-4c28-ab45-7912564ed055" containerName="maas-api"
Jul 28 01:42:38.431859 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.431769    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0c2aa7f-403d-4f1e-8643-db36a47df246" containerName="main"
Jul 28 01:42:38.431859 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.431779    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0c2aa7f-403d-4f1e-8643-db36a47df246" containerName="main"
Jul 28 01:42:38.431974 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.431960    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0c2aa7f-403d-4f1e-8643-db36a47df246" containerName="main"
Jul 28 01:42:38.432108 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.432092    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c81907ea-5b5d-4c28-ab45-7912564ed055" containerName="maas-api"
Jul 28 01:42:38.436702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.436673    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:42:38.442946 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.442929    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-45a9e6-serving-cert\""
Jul 28 01:42:38.459030 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.459007    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"]
Jul 28 01:42:38.550127 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.550093    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-maas-api-tls\") pod \"maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz\" (UID: \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:42:38.550316 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.550229    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klsvv\" (UniqueName: \"kubernetes.io/projected/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-kube-api-access-klsvv\") pod \"maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz\" (UID: \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:42:38.550375 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.550330    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz\" (UID: \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:42:38.651733 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.651696    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klsvv\" (UniqueName: \"kubernetes.io/projected/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-kube-api-access-klsvv\") pod \"maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz\" (UID: \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:42:38.651907 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.651774    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz\" (UID: \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:42:38.651907 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.651823    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-maas-api-tls\") pod \"maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz\" (UID: \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:42:38.651989 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:42:38.651967    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-serving-cert: secret "maas-api-e2e-inf-a-45a9e6-serving-cert" not found
Jul 28 01:42:38.652072 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:42:38.652059    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-maas-api-tls podName:3e3b7bbf-9639-45ca-b94a-4dfc811296a9 nodeName:}" failed. No retries permitted until 2026-07-28 01:42:39.152033364 +0000 UTC m=+2310.196927330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-maas-api-tls") pod "maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz" (UID: "3e3b7bbf-9639-45ca-b94a-4dfc811296a9") : secret "maas-api-e2e-inf-a-45a9e6-serving-cert" not found
Jul 28 01:42:38.652458 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.652441    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-openshift-service-ca\") pod \"maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz\" (UID: \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:42:38.677565 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:38.677540    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klsvv\" (UniqueName: \"kubernetes.io/projected/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-kube-api-access-klsvv\") pod \"maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz\" (UID: \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:42:39.157414 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:39.157372    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-maas-api-tls\") pod \"maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz\" (UID: \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:42:39.160303 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:39.160265    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-maas-api-tls\") pod \"maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz\" (UID: \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:42:39.349002 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:39.348965    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:42:39.485191 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:39.485158    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"]
Jul 28 01:42:39.485916 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:42:39.485884    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e3b7bbf_9639_45ca_b94a_4dfc811296a9.slice/crio-bd4131b411e2519259b08e3c70fff8a3c200dfaa860374b583577596850076da WatchSource:0}: Error finding container bd4131b411e2519259b08e3c70fff8a3c200dfaa860374b583577596850076da: Status 404 returned error can't find the container with id bd4131b411e2519259b08e3c70fff8a3c200dfaa860374b583577596850076da
Jul 28 01:42:39.696857 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:39.696774    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz" event={"ID":"3e3b7bbf-9639-45ca-b94a-4dfc811296a9","Type":"ContainerStarted","Data":"bd4131b411e2519259b08e3c70fff8a3c200dfaa860374b583577596850076da"}
Jul 28 01:42:40.703014 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:40.702979    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz" event={"ID":"3e3b7bbf-9639-45ca-b94a-4dfc811296a9","Type":"ContainerStarted","Data":"c0a381c42cdba385c0e4e1a4e4e48109dfb7c33a1514c21028d7708e533f0e9e"}
Jul 28 01:42:40.703387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:40.703093    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:42:40.720097 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:40.720045    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz" podStartSLOduration=2.25608229 podStartE2EDuration="2.720031358s" totalImagesPullingTime="463.949068ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:42:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:42:40.719424356 +0000 UTC m=+2311.764318325" watchObservedRunningTime="2026-07-28 01:42:40.720031358 +0000 UTC m=+2311.764925346"
Jul 28 01:42:46.713522 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:42:46.713471    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:43:06.658611 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:06.658526    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"]
Jul 28 01:43:06.662759 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:06.662730    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:43:06.665826 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:06.665801    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a5bff2-serving-cert\""
Jul 28 01:43:06.680632 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:06.680611    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"]
Jul 28 01:43:06.696204 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:06.696172    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bnxp\" (UniqueName: \"kubernetes.io/projected/bd8d93b0-1436-4673-ae64-ae7b144158a1-kube-api-access-8bnxp\") pod \"maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw\" (UID: \"bd8d93b0-1436-4673-ae64-ae7b144158a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:43:06.696331 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:06.696308    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd8d93b0-1436-4673-ae64-ae7b144158a1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw\" (UID: \"bd8d93b0-1436-4673-ae64-ae7b144158a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:43:06.696397 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:06.696381    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd8d93b0-1436-4673-ae64-ae7b144158a1-maas-api-tls\") pod \"maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw\" (UID: \"bd8d93b0-1436-4673-ae64-ae7b144158a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:43:06.797013 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:06.796984    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd8d93b0-1436-4673-ae64-ae7b144158a1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw\" (UID: \"bd8d93b0-1436-4673-ae64-ae7b144158a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:43:06.797170 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:06.797027    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd8d93b0-1436-4673-ae64-ae7b144158a1-maas-api-tls\") pod \"maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw\" (UID: \"bd8d93b0-1436-4673-ae64-ae7b144158a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:43:06.797170 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:06.797091    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bnxp\" (UniqueName: \"kubernetes.io/projected/bd8d93b0-1436-4673-ae64-ae7b144158a1-kube-api-access-8bnxp\") pod \"maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw\" (UID: \"bd8d93b0-1436-4673-ae64-ae7b144158a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:43:06.797247 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:43:06.797197    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-serving-cert: secret "maas-api-e2e-inf-b-a5bff2-serving-cert" not found
Jul 28 01:43:06.797289 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:43:06.797278    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd8d93b0-1436-4673-ae64-ae7b144158a1-maas-api-tls podName:bd8d93b0-1436-4673-ae64-ae7b144158a1 nodeName:}" failed. No retries permitted until 2026-07-28 01:43:07.297262286 +0000 UTC m=+2338.342156231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bd8d93b0-1436-4673-ae64-ae7b144158a1-maas-api-tls") pod "maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw" (UID: "bd8d93b0-1436-4673-ae64-ae7b144158a1") : secret "maas-api-e2e-inf-b-a5bff2-serving-cert" not found
Jul 28 01:43:06.797702 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:06.797683    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd8d93b0-1436-4673-ae64-ae7b144158a1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw\" (UID: \"bd8d93b0-1436-4673-ae64-ae7b144158a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:43:06.814847 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:06.814816    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bnxp\" (UniqueName: \"kubernetes.io/projected/bd8d93b0-1436-4673-ae64-ae7b144158a1-kube-api-access-8bnxp\") pod \"maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw\" (UID: \"bd8d93b0-1436-4673-ae64-ae7b144158a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:43:07.302968 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:07.302934    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd8d93b0-1436-4673-ae64-ae7b144158a1-maas-api-tls\") pod \"maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw\" (UID: \"bd8d93b0-1436-4673-ae64-ae7b144158a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:43:07.305426 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:07.305405    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd8d93b0-1436-4673-ae64-ae7b144158a1-maas-api-tls\") pod \"maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw\" (UID: \"bd8d93b0-1436-4673-ae64-ae7b144158a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:43:07.576319 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:07.576283    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:43:07.715862 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:07.715838    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"]
Jul 28 01:43:07.718065 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:43:07.718036    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd8d93b0_1436_4673_ae64_ae7b144158a1.slice/crio-61f18f231dc8911ab814306419eaacb4a6344bc0f854f032b73871c4f34c096e WatchSource:0}: Error finding container 61f18f231dc8911ab814306419eaacb4a6344bc0f854f032b73871c4f34c096e: Status 404 returned error can't find the container with id 61f18f231dc8911ab814306419eaacb4a6344bc0f854f032b73871c4f34c096e
Jul 28 01:43:07.834601 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:07.834521    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw" event={"ID":"bd8d93b0-1436-4673-ae64-ae7b144158a1","Type":"ContainerStarted","Data":"61f18f231dc8911ab814306419eaacb4a6344bc0f854f032b73871c4f34c096e"}
Jul 28 01:43:08.842000 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:08.841964    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw" event={"ID":"bd8d93b0-1436-4673-ae64-ae7b144158a1","Type":"ContainerStarted","Data":"efa52c469ffc67bbf08f04c3b7a1d652f25f88fcdbdf264e20eee7672fb15aca"}
Jul 28 01:43:08.842394 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:08.842061    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:43:08.861062 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:08.861017    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw" podStartSLOduration=2.24104241 podStartE2EDuration="2.861002761s" totalImagesPullingTime="619.960351ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:43:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:43:08.859378426 +0000 UTC m=+2339.904272394" watchObservedRunningTime="2026-07-28 01:43:08.861002761 +0000 UTC m=+2339.905896726"
Jul 28 01:43:14.852316 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:14.852289    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:43:59.402759 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.402729    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"]
Jul 28 01:43:59.406017 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.406000    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.409261 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.409241    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a5bff2\"/\"kube-root-ca.crt\""
Jul 28 01:43:59.410228 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.410205    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a5bff2\"/\"default-dockercfg-4p5jc\""
Jul 28 01:43:59.410359 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.410341    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a5bff2\"/\"test-model-a5bff2-kserve-self-signed-certs\""
Jul 28 01:43:59.410568 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.410546    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a5bff2\"/\"openshift-service-ca.crt\""
Jul 28 01:43:59.415227 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.415208    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"]
Jul 28 01:43:59.467402 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.467376    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-dshm\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.467528 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.467416    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz8j4\" (UniqueName: \"kubernetes.io/projected/47707c3d-91ca-4219-8bea-cb4389085982-kube-api-access-bz8j4\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.467592 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.467545    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-model-cache\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.467592 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.467566    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-tmp-dir\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.467667 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.467657    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47707c3d-91ca-4219-8bea-cb4389085982-tls-certs\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.467712 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.467700    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-home\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.568435 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.568408    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-model-cache\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.568569 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.568439    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-tmp-dir\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.568569 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.568500    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47707c3d-91ca-4219-8bea-cb4389085982-tls-certs\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.568569 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.568528    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-home\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.568569 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.568562    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-dshm\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.568765 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.568577    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz8j4\" (UniqueName: \"kubernetes.io/projected/47707c3d-91ca-4219-8bea-cb4389085982-kube-api-access-bz8j4\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.568883 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.568844    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-model-cache\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.568883 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.568859    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-tmp-dir\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.569032 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.568913    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-home\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.570933 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.570910    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47707c3d-91ca-4219-8bea-cb4389085982-tls-certs\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.571050 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.570933    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-dshm\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.577374 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.577354    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz8j4\" (UniqueName: \"kubernetes.io/projected/47707c3d-91ca-4219-8bea-cb4389085982-kube-api-access-bz8j4\") pod \"test-model-a5bff2-kserve-85dc6f7565-ckdmh\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") " pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.718194 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.718129    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:43:59.851257 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:43:59.851217    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"]
Jul 28 01:43:59.853717 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:43:59.853688    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47707c3d_91ca_4219_8bea_cb4389085982.slice/crio-601694be257f68b2c8a995c877f9c232166c6b3b75a77e434601d0691182cdf0 WatchSource:0}: Error finding container 601694be257f68b2c8a995c877f9c232166c6b3b75a77e434601d0691182cdf0: Status 404 returned error can't find the container with id 601694be257f68b2c8a995c877f9c232166c6b3b75a77e434601d0691182cdf0
Jul 28 01:44:00.085641 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:00.085602    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh" event={"ID":"47707c3d-91ca-4219-8bea-cb4389085982","Type":"ContainerStarted","Data":"f7b5166e7c376112e4545ce636c091d6ae1da3fd207e0165371f4795cb9ab75c"}
Jul 28 01:44:00.085800 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:00.085650    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh" event={"ID":"47707c3d-91ca-4219-8bea-cb4389085982","Type":"ContainerStarted","Data":"601694be257f68b2c8a995c877f9c232166c6b3b75a77e434601d0691182cdf0"}
Jul 28 01:44:00.104503 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:00.104438    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh" podStartSLOduration=1.104420197 podStartE2EDuration="1.104420197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:43:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:44:00.101929171 +0000 UTC m=+2391.146823141" watchObservedRunningTime="2026-07-28 01:44:00.104420197 +0000 UTC m=+2391.149314163"
Jul 28 01:44:09.718915 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:09.718883    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:44:09.719560 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:09.718932    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:44:09.731241 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:09.731217    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:44:10.147551 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:10.147524    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:44:10.223895 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:10.223782    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:44:10.230197 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:10.224593    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:44:10.233812 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:10.233788    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:44:10.233948 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:10.233880    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:44:10.234670 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:10.234652    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:44:10.240576 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:10.240560    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:44:41.738588 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:41.738551    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"]
Jul 28 01:44:52.452816 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:52.452779    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"]
Jul 28 01:44:52.453221 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:52.453024    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz" podUID="3e3b7bbf-9639-45ca-b94a-4dfc811296a9" containerName="maas-api" containerID="cri-o://c0a381c42cdba385c0e4e1a4e4e48109dfb7c33a1514c21028d7708e533f0e9e" gracePeriod=30
Jul 28 01:44:52.702177 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:52.702151    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:44:52.856623 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:52.856597    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klsvv\" (UniqueName: \"kubernetes.io/projected/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-kube-api-access-klsvv\") pod \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\" (UID: \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\") "
Jul 28 01:44:52.856801 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:52.856634    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-maas-api-tls\") pod \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\" (UID: \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\") "
Jul 28 01:44:52.856801 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:52.856698    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-openshift-service-ca\") pod \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\" (UID: \"3e3b7bbf-9639-45ca-b94a-4dfc811296a9\") "
Jul 28 01:44:52.857161 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:52.857131    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3e3b7bbf-9639-45ca-b94a-4dfc811296a9" (UID: "3e3b7bbf-9639-45ca-b94a-4dfc811296a9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:44:52.858812 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:52.858786    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-kube-api-access-klsvv" (OuterVolumeSpecName: "kube-api-access-klsvv") pod "3e3b7bbf-9639-45ca-b94a-4dfc811296a9" (UID: "3e3b7bbf-9639-45ca-b94a-4dfc811296a9"). InnerVolumeSpecName "kube-api-access-klsvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:44:52.858897 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:52.858830    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3e3b7bbf-9639-45ca-b94a-4dfc811296a9" (UID: "3e3b7bbf-9639-45ca-b94a-4dfc811296a9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:44:52.957992 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:52.957964    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klsvv\" (UniqueName: \"kubernetes.io/projected/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-kube-api-access-klsvv\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:44:52.957992 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:52.957988    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:44:52.957992 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:52.957997    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3b7bbf-9639-45ca-b94a-4dfc811296a9-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:44:53.341402 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:53.341370    2565 generic.go:356] "Generic (PLEG): container finished" podID="3e3b7bbf-9639-45ca-b94a-4dfc811296a9" containerID="c0a381c42cdba385c0e4e1a4e4e48109dfb7c33a1514c21028d7708e533f0e9e" exitCode=0
Jul 28 01:44:53.341586 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:53.341410    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"
Jul 28 01:44:53.341586 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:53.341470    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz" event={"ID":"3e3b7bbf-9639-45ca-b94a-4dfc811296a9","Type":"ContainerDied","Data":"c0a381c42cdba385c0e4e1a4e4e48109dfb7c33a1514c21028d7708e533f0e9e"}
Jul 28 01:44:53.341586 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:53.341540    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz" event={"ID":"3e3b7bbf-9639-45ca-b94a-4dfc811296a9","Type":"ContainerDied","Data":"bd4131b411e2519259b08e3c70fff8a3c200dfaa860374b583577596850076da"}
Jul 28 01:44:53.341586 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:53.341565    2565 scope.go:117] "RemoveContainer" containerID="c0a381c42cdba385c0e4e1a4e4e48109dfb7c33a1514c21028d7708e533f0e9e"
Jul 28 01:44:53.351345 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:53.351329    2565 scope.go:117] "RemoveContainer" containerID="c0a381c42cdba385c0e4e1a4e4e48109dfb7c33a1514c21028d7708e533f0e9e"
Jul 28 01:44:53.351626 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:44:53.351609    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0a381c42cdba385c0e4e1a4e4e48109dfb7c33a1514c21028d7708e533f0e9e\": container with ID starting with c0a381c42cdba385c0e4e1a4e4e48109dfb7c33a1514c21028d7708e533f0e9e not found: ID does not exist" containerID="c0a381c42cdba385c0e4e1a4e4e48109dfb7c33a1514c21028d7708e533f0e9e"
Jul 28 01:44:53.351689 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:53.351636    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0a381c42cdba385c0e4e1a4e4e48109dfb7c33a1514c21028d7708e533f0e9e"} err="failed to get container status \"c0a381c42cdba385c0e4e1a4e4e48109dfb7c33a1514c21028d7708e533f0e9e\": rpc error: code = NotFound desc = could not find container \"c0a381c42cdba385c0e4e1a4e4e48109dfb7c33a1514c21028d7708e533f0e9e\": container with ID starting with c0a381c42cdba385c0e4e1a4e4e48109dfb7c33a1514c21028d7708e533f0e9e not found: ID does not exist"
Jul 28 01:44:53.362703 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:53.362683    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"]
Jul 28 01:44:53.366476 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:53.366458    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-45a9e6-84fb656574-z4bnz"]
Jul 28 01:44:53.467178 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:53.467150    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e3b7bbf-9639-45ca-b94a-4dfc811296a9" path="/var/lib/kubelet/pods/3e3b7bbf-9639-45ca-b94a-4dfc811296a9/volumes"
Jul 28 01:44:56.758138 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:56.758079    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh" podUID="47707c3d-91ca-4219-8bea-cb4389085982" containerName="main" containerID="cri-o://f7b5166e7c376112e4545ce636c091d6ae1da3fd207e0165371f4795cb9ab75c" gracePeriod=45
Jul 28 01:44:57.009180 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.009126    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:44:57.198800 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.198767    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-tmp-dir\") pod \"47707c3d-91ca-4219-8bea-cb4389085982\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") "
Jul 28 01:44:57.198800 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.198805    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-dshm\") pod \"47707c3d-91ca-4219-8bea-cb4389085982\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") "
Jul 28 01:44:57.199107 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.198822    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-model-cache\") pod \"47707c3d-91ca-4219-8bea-cb4389085982\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") "
Jul 28 01:44:57.199107 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.198879    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz8j4\" (UniqueName: \"kubernetes.io/projected/47707c3d-91ca-4219-8bea-cb4389085982-kube-api-access-bz8j4\") pod \"47707c3d-91ca-4219-8bea-cb4389085982\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") "
Jul 28 01:44:57.199107 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.198911    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47707c3d-91ca-4219-8bea-cb4389085982-tls-certs\") pod \"47707c3d-91ca-4219-8bea-cb4389085982\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") "
Jul 28 01:44:57.199107 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.198954    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-home\") pod \"47707c3d-91ca-4219-8bea-cb4389085982\" (UID: \"47707c3d-91ca-4219-8bea-cb4389085982\") "
Jul 28 01:44:57.199107 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.199082    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "47707c3d-91ca-4219-8bea-cb4389085982" (UID: "47707c3d-91ca-4219-8bea-cb4389085982"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:44:57.199335 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.199139    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-model-cache" (OuterVolumeSpecName: "model-cache") pod "47707c3d-91ca-4219-8bea-cb4389085982" (UID: "47707c3d-91ca-4219-8bea-cb4389085982"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:44:57.199335 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.199249    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-home" (OuterVolumeSpecName: "home") pod "47707c3d-91ca-4219-8bea-cb4389085982" (UID: "47707c3d-91ca-4219-8bea-cb4389085982"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:44:57.199335 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.199315    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-tmp-dir\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:44:57.199335 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.199330    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-model-cache\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:44:57.201299 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.201270    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-dshm" (OuterVolumeSpecName: "dshm") pod "47707c3d-91ca-4219-8bea-cb4389085982" (UID: "47707c3d-91ca-4219-8bea-cb4389085982"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:44:57.201299 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.201276    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47707c3d-91ca-4219-8bea-cb4389085982-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "47707c3d-91ca-4219-8bea-cb4389085982" (UID: "47707c3d-91ca-4219-8bea-cb4389085982"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:44:57.201446 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.201312    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47707c3d-91ca-4219-8bea-cb4389085982-kube-api-access-bz8j4" (OuterVolumeSpecName: "kube-api-access-bz8j4") pod "47707c3d-91ca-4219-8bea-cb4389085982" (UID: "47707c3d-91ca-4219-8bea-cb4389085982"). InnerVolumeSpecName "kube-api-access-bz8j4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:44:57.299941 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.299867    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-dshm\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:44:57.299941 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.299897    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz8j4\" (UniqueName: \"kubernetes.io/projected/47707c3d-91ca-4219-8bea-cb4389085982-kube-api-access-bz8j4\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:44:57.299941 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.299912    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47707c3d-91ca-4219-8bea-cb4389085982-tls-certs\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:44:57.299941 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.299924    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47707c3d-91ca-4219-8bea-cb4389085982-home\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:44:57.361246 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.361211    2565 generic.go:356] "Generic (PLEG): container finished" podID="47707c3d-91ca-4219-8bea-cb4389085982" containerID="f7b5166e7c376112e4545ce636c091d6ae1da3fd207e0165371f4795cb9ab75c" exitCode=0
Jul 28 01:44:57.361390 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.361257    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"
Jul 28 01:44:57.361390 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.361300    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh" event={"ID":"47707c3d-91ca-4219-8bea-cb4389085982","Type":"ContainerDied","Data":"f7b5166e7c376112e4545ce636c091d6ae1da3fd207e0165371f4795cb9ab75c"}
Jul 28 01:44:57.361390 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.361357    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh" event={"ID":"47707c3d-91ca-4219-8bea-cb4389085982","Type":"ContainerDied","Data":"601694be257f68b2c8a995c877f9c232166c6b3b75a77e434601d0691182cdf0"}
Jul 28 01:44:57.361390 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.361378    2565 scope.go:117] "RemoveContainer" containerID="f7b5166e7c376112e4545ce636c091d6ae1da3fd207e0165371f4795cb9ab75c"
Jul 28 01:44:57.370430 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.370413    2565 scope.go:117] "RemoveContainer" containerID="f7b5166e7c376112e4545ce636c091d6ae1da3fd207e0165371f4795cb9ab75c"
Jul 28 01:44:57.370888 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:44:57.370868    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7b5166e7c376112e4545ce636c091d6ae1da3fd207e0165371f4795cb9ab75c\": container with ID starting with f7b5166e7c376112e4545ce636c091d6ae1da3fd207e0165371f4795cb9ab75c not found: ID does not exist" containerID="f7b5166e7c376112e4545ce636c091d6ae1da3fd207e0165371f4795cb9ab75c"
Jul 28 01:44:57.370984 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.370896    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b5166e7c376112e4545ce636c091d6ae1da3fd207e0165371f4795cb9ab75c"} err="failed to get container status \"f7b5166e7c376112e4545ce636c091d6ae1da3fd207e0165371f4795cb9ab75c\": rpc error: code = NotFound desc = could not find container \"f7b5166e7c376112e4545ce636c091d6ae1da3fd207e0165371f4795cb9ab75c\": container with ID starting with f7b5166e7c376112e4545ce636c091d6ae1da3fd207e0165371f4795cb9ab75c not found: ID does not exist"
Jul 28 01:44:57.381562 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.381531    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"]
Jul 28 01:44:57.385242 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.385220    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a5bff2/test-model-a5bff2-kserve-85dc6f7565-ckdmh"]
Jul 28 01:44:57.466585 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:44:57.466564    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47707c3d-91ca-4219-8bea-cb4389085982" path="/var/lib/kubelet/pods/47707c3d-91ca-4219-8bea-cb4389085982/volumes"
Jul 28 01:45:10.153823 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.153792    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"]
Jul 28 01:45:10.154237 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.154012    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw" podUID="bd8d93b0-1436-4673-ae64-ae7b144158a1" containerName="maas-api" containerID="cri-o://efa52c469ffc67bbf08f04c3b7a1d652f25f88fcdbdf264e20eee7672fb15aca" gracePeriod=30
Jul 28 01:45:10.406185 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.406130    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:45:10.422940 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.422911    2565 generic.go:356] "Generic (PLEG): container finished" podID="bd8d93b0-1436-4673-ae64-ae7b144158a1" containerID="efa52c469ffc67bbf08f04c3b7a1d652f25f88fcdbdf264e20eee7672fb15aca" exitCode=0
Jul 28 01:45:10.423054 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.422947    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"
Jul 28 01:45:10.423054 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.422965    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw" event={"ID":"bd8d93b0-1436-4673-ae64-ae7b144158a1","Type":"ContainerDied","Data":"efa52c469ffc67bbf08f04c3b7a1d652f25f88fcdbdf264e20eee7672fb15aca"}
Jul 28 01:45:10.423054 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.423013    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw" event={"ID":"bd8d93b0-1436-4673-ae64-ae7b144158a1","Type":"ContainerDied","Data":"61f18f231dc8911ab814306419eaacb4a6344bc0f854f032b73871c4f34c096e"}
Jul 28 01:45:10.423054 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.423039    2565 scope.go:117] "RemoveContainer" containerID="efa52c469ffc67bbf08f04c3b7a1d652f25f88fcdbdf264e20eee7672fb15aca"
Jul 28 01:45:10.433302 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.433282    2565 scope.go:117] "RemoveContainer" containerID="efa52c469ffc67bbf08f04c3b7a1d652f25f88fcdbdf264e20eee7672fb15aca"
Jul 28 01:45:10.433571 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:45:10.433549    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efa52c469ffc67bbf08f04c3b7a1d652f25f88fcdbdf264e20eee7672fb15aca\": container with ID starting with efa52c469ffc67bbf08f04c3b7a1d652f25f88fcdbdf264e20eee7672fb15aca not found: ID does not exist" containerID="efa52c469ffc67bbf08f04c3b7a1d652f25f88fcdbdf264e20eee7672fb15aca"
Jul 28 01:45:10.433644 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.433580    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efa52c469ffc67bbf08f04c3b7a1d652f25f88fcdbdf264e20eee7672fb15aca"} err="failed to get container status \"efa52c469ffc67bbf08f04c3b7a1d652f25f88fcdbdf264e20eee7672fb15aca\": rpc error: code = NotFound desc = could not find container \"efa52c469ffc67bbf08f04c3b7a1d652f25f88fcdbdf264e20eee7672fb15aca\": container with ID starting with efa52c469ffc67bbf08f04c3b7a1d652f25f88fcdbdf264e20eee7672fb15aca not found: ID does not exist"
Jul 28 01:45:10.515920 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.515895    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd8d93b0-1436-4673-ae64-ae7b144158a1-openshift-service-ca\") pod \"bd8d93b0-1436-4673-ae64-ae7b144158a1\" (UID: \"bd8d93b0-1436-4673-ae64-ae7b144158a1\") "
Jul 28 01:45:10.516049 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.515928    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bnxp\" (UniqueName: \"kubernetes.io/projected/bd8d93b0-1436-4673-ae64-ae7b144158a1-kube-api-access-8bnxp\") pod \"bd8d93b0-1436-4673-ae64-ae7b144158a1\" (UID: \"bd8d93b0-1436-4673-ae64-ae7b144158a1\") "
Jul 28 01:45:10.516049 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.515979    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd8d93b0-1436-4673-ae64-ae7b144158a1-maas-api-tls\") pod \"bd8d93b0-1436-4673-ae64-ae7b144158a1\" (UID: \"bd8d93b0-1436-4673-ae64-ae7b144158a1\") "
Jul 28 01:45:10.516338 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.516305    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd8d93b0-1436-4673-ae64-ae7b144158a1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd8d93b0-1436-4673-ae64-ae7b144158a1" (UID: "bd8d93b0-1436-4673-ae64-ae7b144158a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:45:10.518260 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.518236    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd8d93b0-1436-4673-ae64-ae7b144158a1-kube-api-access-8bnxp" (OuterVolumeSpecName: "kube-api-access-8bnxp") pod "bd8d93b0-1436-4673-ae64-ae7b144158a1" (UID: "bd8d93b0-1436-4673-ae64-ae7b144158a1"). InnerVolumeSpecName "kube-api-access-8bnxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:45:10.518427 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.518403    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd8d93b0-1436-4673-ae64-ae7b144158a1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd8d93b0-1436-4673-ae64-ae7b144158a1" (UID: "bd8d93b0-1436-4673-ae64-ae7b144158a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:45:10.617168 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.617140    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bnxp\" (UniqueName: \"kubernetes.io/projected/bd8d93b0-1436-4673-ae64-ae7b144158a1-kube-api-access-8bnxp\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:45:10.617168 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.617167    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd8d93b0-1436-4673-ae64-ae7b144158a1-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:45:10.617341 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.617179    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd8d93b0-1436-4673-ae64-ae7b144158a1-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:45:10.745326 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.745297    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"]
Jul 28 01:45:10.746949 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:10.746930    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a5bff2-5cd868c889-rzzhw"]
Jul 28 01:45:11.466283 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:11.466249    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd8d93b0-1436-4673-ae64-ae7b144158a1" path="/var/lib/kubelet/pods/bd8d93b0-1436-4673-ae64-ae7b144158a1/volumes"
Jul 28 01:45:20.211287 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.211251    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"]
Jul 28 01:45:20.211871 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.211854    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3b7bbf-9639-45ca-b94a-4dfc811296a9" containerName="maas-api"
Jul 28 01:45:20.211920 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.211875    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3b7bbf-9639-45ca-b94a-4dfc811296a9" containerName="maas-api"
Jul 28 01:45:20.211920 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.211889    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47707c3d-91ca-4219-8bea-cb4389085982" containerName="main"
Jul 28 01:45:20.211920 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.211897    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="47707c3d-91ca-4219-8bea-cb4389085982" containerName="main"
Jul 28 01:45:20.211920 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.211914    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd8d93b0-1436-4673-ae64-ae7b144158a1" containerName="maas-api"
Jul 28 01:45:20.212057 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.211922    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd8d93b0-1436-4673-ae64-ae7b144158a1" containerName="maas-api"
Jul 28 01:45:20.212057 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.212008    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd8d93b0-1436-4673-ae64-ae7b144158a1" containerName="maas-api"
Jul 28 01:45:20.212057 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.212022    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="47707c3d-91ca-4219-8bea-cb4389085982" containerName="main"
Jul 28 01:45:20.212057 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.212033    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3b7bbf-9639-45ca-b94a-4dfc811296a9" containerName="maas-api"
Jul 28 01:45:20.215556 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.215527    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:45:20.218349 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.218316    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ea62b7-serving-cert\""
Jul 28 01:45:20.227547 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.227524    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"]
Jul 28 01:45:20.305662 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.305623    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd9d5\" (UniqueName: \"kubernetes.io/projected/5b654ead-fab7-4322-bdbf-902a9b4daefa-kube-api-access-fd9d5\") pod \"maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp\" (UID: \"5b654ead-fab7-4322-bdbf-902a9b4daefa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:45:20.305838 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.305692    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b654ead-fab7-4322-bdbf-902a9b4daefa-openshift-service-ca\") pod \"maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp\" (UID: \"5b654ead-fab7-4322-bdbf-902a9b4daefa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:45:20.305838 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.305772    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b654ead-fab7-4322-bdbf-902a9b4daefa-maas-api-tls\") pod \"maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp\" (UID: \"5b654ead-fab7-4322-bdbf-902a9b4daefa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:45:20.407059 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.407024    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd9d5\" (UniqueName: \"kubernetes.io/projected/5b654ead-fab7-4322-bdbf-902a9b4daefa-kube-api-access-fd9d5\") pod \"maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp\" (UID: \"5b654ead-fab7-4322-bdbf-902a9b4daefa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:45:20.407202 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.407084    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b654ead-fab7-4322-bdbf-902a9b4daefa-openshift-service-ca\") pod \"maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp\" (UID: \"5b654ead-fab7-4322-bdbf-902a9b4daefa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:45:20.407202 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.407140    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b654ead-fab7-4322-bdbf-902a9b4daefa-maas-api-tls\") pod \"maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp\" (UID: \"5b654ead-fab7-4322-bdbf-902a9b4daefa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:45:20.407274 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:45:20.407239    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-serving-cert: secret "maas-api-e2e-api-a-ea62b7-serving-cert" not found
Jul 28 01:45:20.407308 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:45:20.407300    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b654ead-fab7-4322-bdbf-902a9b4daefa-maas-api-tls podName:5b654ead-fab7-4322-bdbf-902a9b4daefa nodeName:}" failed. No retries permitted until 2026-07-28 01:45:20.907283315 +0000 UTC m=+2471.952177261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5b654ead-fab7-4322-bdbf-902a9b4daefa-maas-api-tls") pod "maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp" (UID: "5b654ead-fab7-4322-bdbf-902a9b4daefa") : secret "maas-api-e2e-api-a-ea62b7-serving-cert" not found
Jul 28 01:45:20.407887 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.407865    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b654ead-fab7-4322-bdbf-902a9b4daefa-openshift-service-ca\") pod \"maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp\" (UID: \"5b654ead-fab7-4322-bdbf-902a9b4daefa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:45:20.428654 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.428622    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd9d5\" (UniqueName: \"kubernetes.io/projected/5b654ead-fab7-4322-bdbf-902a9b4daefa-kube-api-access-fd9d5\") pod \"maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp\" (UID: \"5b654ead-fab7-4322-bdbf-902a9b4daefa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:45:20.912698 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.912665    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b654ead-fab7-4322-bdbf-902a9b4daefa-maas-api-tls\") pod \"maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp\" (UID: \"5b654ead-fab7-4322-bdbf-902a9b4daefa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:45:20.915212 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:20.915185    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b654ead-fab7-4322-bdbf-902a9b4daefa-maas-api-tls\") pod \"maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp\" (UID: \"5b654ead-fab7-4322-bdbf-902a9b4daefa\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:45:21.129461 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:21.129429    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:45:21.263917 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:21.263888    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"]
Jul 28 01:45:21.265864 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:45:21.265831    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b654ead_fab7_4322_bdbf_902a9b4daefa.slice/crio-70a77f4bf061050b025d804822575593728c0b757238832cc689db1693ad237f WatchSource:0}: Error finding container 70a77f4bf061050b025d804822575593728c0b757238832cc689db1693ad237f: Status 404 returned error can't find the container with id 70a77f4bf061050b025d804822575593728c0b757238832cc689db1693ad237f
Jul 28 01:45:21.267241 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:21.267222    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:45:21.478619 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:21.478549    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp" event={"ID":"5b654ead-fab7-4322-bdbf-902a9b4daefa","Type":"ContainerStarted","Data":"70a77f4bf061050b025d804822575593728c0b757238832cc689db1693ad237f"}
Jul 28 01:45:22.485353 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:22.485315    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp" event={"ID":"5b654ead-fab7-4322-bdbf-902a9b4daefa","Type":"ContainerStarted","Data":"7976798f9e0143ff38d57ccdf3e071b91f934967132d63293faefb2873cf865e"}
Jul 28 01:45:22.485742 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:22.485421    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:45:22.505017 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:22.504964    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp" podStartSLOduration=2.077653873 podStartE2EDuration="2.5049464s" totalImagesPullingTime="427.292527ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:45:22.502905284 +0000 UTC m=+2473.547799252" watchObservedRunningTime="2026-07-28 01:45:22.5049464 +0000 UTC m=+2473.549840366"
Jul 28 01:45:28.496808 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:28.496777    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:45:49.411817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:49.411781    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"]
Jul 28 01:45:49.415285 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:49.415264    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:45:49.418844 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:49.418822    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-5e7618-serving-cert\""
Jul 28 01:45:49.427357 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:49.427336    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"]
Jul 28 01:45:49.473728 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:49.473690    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9d10b2c-48aa-4981-8111-c23cd633ab4a-openshift-service-ca\") pod \"maas-api-e2e-api-b-5e7618-669d4d54dd-99r47\" (UID: \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:45:49.473896 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:49.473734    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z9d2\" (UniqueName: \"kubernetes.io/projected/a9d10b2c-48aa-4981-8111-c23cd633ab4a-kube-api-access-8z9d2\") pod \"maas-api-e2e-api-b-5e7618-669d4d54dd-99r47\" (UID: \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:45:49.474109 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:49.474084    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9d10b2c-48aa-4981-8111-c23cd633ab4a-maas-api-tls\") pod \"maas-api-e2e-api-b-5e7618-669d4d54dd-99r47\" (UID: \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:45:49.575207 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:49.575177    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9d10b2c-48aa-4981-8111-c23cd633ab4a-maas-api-tls\") pod \"maas-api-e2e-api-b-5e7618-669d4d54dd-99r47\" (UID: \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:45:49.575412 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:49.575303    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9d10b2c-48aa-4981-8111-c23cd633ab4a-openshift-service-ca\") pod \"maas-api-e2e-api-b-5e7618-669d4d54dd-99r47\" (UID: \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:45:49.575412 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:45:49.575329    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-serving-cert: secret "maas-api-e2e-api-b-5e7618-serving-cert" not found
Jul 28 01:45:49.575412 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:49.575333    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z9d2\" (UniqueName: \"kubernetes.io/projected/a9d10b2c-48aa-4981-8111-c23cd633ab4a-kube-api-access-8z9d2\") pod \"maas-api-e2e-api-b-5e7618-669d4d54dd-99r47\" (UID: \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:45:49.575412 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:45:49.575395    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9d10b2c-48aa-4981-8111-c23cd633ab4a-maas-api-tls podName:a9d10b2c-48aa-4981-8111-c23cd633ab4a nodeName:}" failed. No retries permitted until 2026-07-28 01:45:50.075374895 +0000 UTC m=+2501.120268842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a9d10b2c-48aa-4981-8111-c23cd633ab4a-maas-api-tls") pod "maas-api-e2e-api-b-5e7618-669d4d54dd-99r47" (UID: "a9d10b2c-48aa-4981-8111-c23cd633ab4a") : secret "maas-api-e2e-api-b-5e7618-serving-cert" not found
Jul 28 01:45:49.576171 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:49.576148    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9d10b2c-48aa-4981-8111-c23cd633ab4a-openshift-service-ca\") pod \"maas-api-e2e-api-b-5e7618-669d4d54dd-99r47\" (UID: \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:45:49.594173 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:49.594147    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z9d2\" (UniqueName: \"kubernetes.io/projected/a9d10b2c-48aa-4981-8111-c23cd633ab4a-kube-api-access-8z9d2\") pod \"maas-api-e2e-api-b-5e7618-669d4d54dd-99r47\" (UID: \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:45:50.079695 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:50.079659    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9d10b2c-48aa-4981-8111-c23cd633ab4a-maas-api-tls\") pod \"maas-api-e2e-api-b-5e7618-669d4d54dd-99r47\" (UID: \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:45:50.082710 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:50.082660    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9d10b2c-48aa-4981-8111-c23cd633ab4a-maas-api-tls\") pod \"maas-api-e2e-api-b-5e7618-669d4d54dd-99r47\" (UID: \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:45:50.326246 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:50.326219    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:45:50.465323 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:50.465299    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"]
Jul 28 01:45:50.466982 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:45:50.466956    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9d10b2c_48aa_4981_8111_c23cd633ab4a.slice/crio-9edd3e5a672eadf7afc8096021475bde35577769e7dd7c1accff9aa147bab86f WatchSource:0}: Error finding container 9edd3e5a672eadf7afc8096021475bde35577769e7dd7c1accff9aa147bab86f: Status 404 returned error can't find the container with id 9edd3e5a672eadf7afc8096021475bde35577769e7dd7c1accff9aa147bab86f
Jul 28 01:45:50.612394 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:50.612308    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47" event={"ID":"a9d10b2c-48aa-4981-8111-c23cd633ab4a","Type":"ContainerStarted","Data":"9edd3e5a672eadf7afc8096021475bde35577769e7dd7c1accff9aa147bab86f"}
Jul 28 01:45:51.618039 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:51.618004    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47" event={"ID":"a9d10b2c-48aa-4981-8111-c23cd633ab4a","Type":"ContainerStarted","Data":"7e52442fa5cc43e0dac4a7d24f2417bd7b6e471a442968b5ae665558771ae7a9"}
Jul 28 01:45:51.618543 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:51.618136    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:45:51.637022 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:51.636973    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47" podStartSLOduration=2.231289759 podStartE2EDuration="2.636958887s" totalImagesPullingTime="405.669128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:45:51.635672747 +0000 UTC m=+2502.680566734" watchObservedRunningTime="2026-07-28 01:45:51.636958887 +0000 UTC m=+2502.681852853"
Jul 28 01:45:57.628132 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:45:57.628107    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:46:32.813044 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:32.813006    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"]
Jul 28 01:46:32.813608 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:32.813248    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp" podUID="5b654ead-fab7-4322-bdbf-902a9b4daefa" containerName="maas-api" containerID="cri-o://7976798f9e0143ff38d57ccdf3e071b91f934967132d63293faefb2873cf865e" gracePeriod=30
Jul 28 01:46:33.064920 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.064865    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:46:33.081870 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.081841    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b654ead-fab7-4322-bdbf-902a9b4daefa-openshift-service-ca\") pod \"5b654ead-fab7-4322-bdbf-902a9b4daefa\" (UID: \"5b654ead-fab7-4322-bdbf-902a9b4daefa\") "
Jul 28 01:46:33.082013 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.081901    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fd9d5\" (UniqueName: \"kubernetes.io/projected/5b654ead-fab7-4322-bdbf-902a9b4daefa-kube-api-access-fd9d5\") pod \"5b654ead-fab7-4322-bdbf-902a9b4daefa\" (UID: \"5b654ead-fab7-4322-bdbf-902a9b4daefa\") "
Jul 28 01:46:33.082013 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.081943    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b654ead-fab7-4322-bdbf-902a9b4daefa-maas-api-tls\") pod \"5b654ead-fab7-4322-bdbf-902a9b4daefa\" (UID: \"5b654ead-fab7-4322-bdbf-902a9b4daefa\") "
Jul 28 01:46:33.082355 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.082318    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b654ead-fab7-4322-bdbf-902a9b4daefa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5b654ead-fab7-4322-bdbf-902a9b4daefa" (UID: "5b654ead-fab7-4322-bdbf-902a9b4daefa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:46:33.085402 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.085379    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b654ead-fab7-4322-bdbf-902a9b4daefa-kube-api-access-fd9d5" (OuterVolumeSpecName: "kube-api-access-fd9d5") pod "5b654ead-fab7-4322-bdbf-902a9b4daefa" (UID: "5b654ead-fab7-4322-bdbf-902a9b4daefa"). InnerVolumeSpecName "kube-api-access-fd9d5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:46:33.085532 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.085439    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b654ead-fab7-4322-bdbf-902a9b4daefa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5b654ead-fab7-4322-bdbf-902a9b4daefa" (UID: "5b654ead-fab7-4322-bdbf-902a9b4daefa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:46:33.182946 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.182912    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b654ead-fab7-4322-bdbf-902a9b4daefa-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:46:33.182946 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.182945    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd9d5\" (UniqueName: \"kubernetes.io/projected/5b654ead-fab7-4322-bdbf-902a9b4daefa-kube-api-access-fd9d5\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:46:33.183176 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.182959    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b654ead-fab7-4322-bdbf-902a9b4daefa-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:46:33.814616 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.814579    2565 generic.go:356] "Generic (PLEG): container finished" podID="5b654ead-fab7-4322-bdbf-902a9b4daefa" containerID="7976798f9e0143ff38d57ccdf3e071b91f934967132d63293faefb2873cf865e" exitCode=0
Jul 28 01:46:33.815042 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.814626    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"
Jul 28 01:46:33.815042 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.814672    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp" event={"ID":"5b654ead-fab7-4322-bdbf-902a9b4daefa","Type":"ContainerDied","Data":"7976798f9e0143ff38d57ccdf3e071b91f934967132d63293faefb2873cf865e"}
Jul 28 01:46:33.815042 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.814717    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp" event={"ID":"5b654ead-fab7-4322-bdbf-902a9b4daefa","Type":"ContainerDied","Data":"70a77f4bf061050b025d804822575593728c0b757238832cc689db1693ad237f"}
Jul 28 01:46:33.815042 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.814738    2565 scope.go:117] "RemoveContainer" containerID="7976798f9e0143ff38d57ccdf3e071b91f934967132d63293faefb2873cf865e"
Jul 28 01:46:33.823537 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.823521    2565 scope.go:117] "RemoveContainer" containerID="7976798f9e0143ff38d57ccdf3e071b91f934967132d63293faefb2873cf865e"
Jul 28 01:46:33.823820 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:46:33.823799    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7976798f9e0143ff38d57ccdf3e071b91f934967132d63293faefb2873cf865e\": container with ID starting with 7976798f9e0143ff38d57ccdf3e071b91f934967132d63293faefb2873cf865e not found: ID does not exist" containerID="7976798f9e0143ff38d57ccdf3e071b91f934967132d63293faefb2873cf865e"
Jul 28 01:46:33.823866 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.823830    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7976798f9e0143ff38d57ccdf3e071b91f934967132d63293faefb2873cf865e"} err="failed to get container status \"7976798f9e0143ff38d57ccdf3e071b91f934967132d63293faefb2873cf865e\": rpc error: code = NotFound desc = could not find container \"7976798f9e0143ff38d57ccdf3e071b91f934967132d63293faefb2873cf865e\": container with ID starting with 7976798f9e0143ff38d57ccdf3e071b91f934967132d63293faefb2873cf865e not found: ID does not exist"
Jul 28 01:46:33.832178 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.832159    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"]
Jul 28 01:46:33.835737 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:33.835718    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ea62b7-79695c8dc6-76tgp"]
Jul 28 01:46:35.472770 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:35.467775    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b654ead-fab7-4322-bdbf-902a9b4daefa" path="/var/lib/kubelet/pods/5b654ead-fab7-4322-bdbf-902a9b4daefa/volumes"
Jul 28 01:46:55.281921 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.281886    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"]
Jul 28 01:46:55.282817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.282102    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47" podUID="a9d10b2c-48aa-4981-8111-c23cd633ab4a" containerName="maas-api" containerID="cri-o://7e52442fa5cc43e0dac4a7d24f2417bd7b6e471a442968b5ae665558771ae7a9" gracePeriod=30
Jul 28 01:46:55.538141 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.538078    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:46:55.693389 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.693358    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z9d2\" (UniqueName: \"kubernetes.io/projected/a9d10b2c-48aa-4981-8111-c23cd633ab4a-kube-api-access-8z9d2\") pod \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\" (UID: \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\") "
Jul 28 01:46:55.693389 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.693394    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9d10b2c-48aa-4981-8111-c23cd633ab4a-openshift-service-ca\") pod \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\" (UID: \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\") "
Jul 28 01:46:55.693595 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.693456    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9d10b2c-48aa-4981-8111-c23cd633ab4a-maas-api-tls\") pod \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\" (UID: \"a9d10b2c-48aa-4981-8111-c23cd633ab4a\") "
Jul 28 01:46:55.693868 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.693838    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9d10b2c-48aa-4981-8111-c23cd633ab4a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a9d10b2c-48aa-4981-8111-c23cd633ab4a" (UID: "a9d10b2c-48aa-4981-8111-c23cd633ab4a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:46:55.695708 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.695683    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9d10b2c-48aa-4981-8111-c23cd633ab4a-kube-api-access-8z9d2" (OuterVolumeSpecName: "kube-api-access-8z9d2") pod "a9d10b2c-48aa-4981-8111-c23cd633ab4a" (UID: "a9d10b2c-48aa-4981-8111-c23cd633ab4a"). InnerVolumeSpecName "kube-api-access-8z9d2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:46:55.695798 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.695684    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9d10b2c-48aa-4981-8111-c23cd633ab4a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a9d10b2c-48aa-4981-8111-c23cd633ab4a" (UID: "a9d10b2c-48aa-4981-8111-c23cd633ab4a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:46:55.794720 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.794639    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z9d2\" (UniqueName: \"kubernetes.io/projected/a9d10b2c-48aa-4981-8111-c23cd633ab4a-kube-api-access-8z9d2\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:46:55.794720 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.794668    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9d10b2c-48aa-4981-8111-c23cd633ab4a-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:46:55.794720 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.794681    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9d10b2c-48aa-4981-8111-c23cd633ab4a-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:46:55.922563 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.922530    2565 generic.go:356] "Generic (PLEG): container finished" podID="a9d10b2c-48aa-4981-8111-c23cd633ab4a" containerID="7e52442fa5cc43e0dac4a7d24f2417bd7b6e471a442968b5ae665558771ae7a9" exitCode=0
Jul 28 01:46:55.922770 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.922576    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47" event={"ID":"a9d10b2c-48aa-4981-8111-c23cd633ab4a","Type":"ContainerDied","Data":"7e52442fa5cc43e0dac4a7d24f2417bd7b6e471a442968b5ae665558771ae7a9"}
Jul 28 01:46:55.922770 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.922591    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"
Jul 28 01:46:55.922770 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.922644    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47" event={"ID":"a9d10b2c-48aa-4981-8111-c23cd633ab4a","Type":"ContainerDied","Data":"9edd3e5a672eadf7afc8096021475bde35577769e7dd7c1accff9aa147bab86f"}
Jul 28 01:46:55.922770 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.922665    2565 scope.go:117] "RemoveContainer" containerID="7e52442fa5cc43e0dac4a7d24f2417bd7b6e471a442968b5ae665558771ae7a9"
Jul 28 01:46:55.932277 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.932263    2565 scope.go:117] "RemoveContainer" containerID="7e52442fa5cc43e0dac4a7d24f2417bd7b6e471a442968b5ae665558771ae7a9"
Jul 28 01:46:55.932528 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:46:55.932509    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e52442fa5cc43e0dac4a7d24f2417bd7b6e471a442968b5ae665558771ae7a9\": container with ID starting with 7e52442fa5cc43e0dac4a7d24f2417bd7b6e471a442968b5ae665558771ae7a9 not found: ID does not exist" containerID="7e52442fa5cc43e0dac4a7d24f2417bd7b6e471a442968b5ae665558771ae7a9"
Jul 28 01:46:55.932579 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.932535    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e52442fa5cc43e0dac4a7d24f2417bd7b6e471a442968b5ae665558771ae7a9"} err="failed to get container status \"7e52442fa5cc43e0dac4a7d24f2417bd7b6e471a442968b5ae665558771ae7a9\": rpc error: code = NotFound desc = could not find container \"7e52442fa5cc43e0dac4a7d24f2417bd7b6e471a442968b5ae665558771ae7a9\": container with ID starting with 7e52442fa5cc43e0dac4a7d24f2417bd7b6e471a442968b5ae665558771ae7a9 not found: ID does not exist"
Jul 28 01:46:55.944114 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.944091    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"]
Jul 28 01:46:55.947970 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:55.947951    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-5e7618-669d4d54dd-99r47"]
Jul 28 01:46:57.466962 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:46:57.466932    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9d10b2c-48aa-4981-8111-c23cd633ab4a" path="/var/lib/kubelet/pods/a9d10b2c-48aa-4981-8111-c23cd633ab4a/volumes"
Jul 28 01:47:05.864124 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:05.864086    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"]
Jul 28 01:47:05.864707 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:05.864651    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9d10b2c-48aa-4981-8111-c23cd633ab4a" containerName="maas-api"
Jul 28 01:47:05.864707 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:05.864671    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9d10b2c-48aa-4981-8111-c23cd633ab4a" containerName="maas-api"
Jul 28 01:47:05.864707 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:05.864702    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b654ead-fab7-4322-bdbf-902a9b4daefa" containerName="maas-api"
Jul 28 01:47:05.864863 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:05.864710    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b654ead-fab7-4322-bdbf-902a9b4daefa" containerName="maas-api"
Jul 28 01:47:05.864863 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:05.864801    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b654ead-fab7-4322-bdbf-902a9b4daefa" containerName="maas-api"
Jul 28 01:47:05.864863 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:05.864815    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9d10b2c-48aa-4981-8111-c23cd633ab4a" containerName="maas-api"
Jul 28 01:47:05.869612 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:05.869591    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:05.872373 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:05.872349    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-0337e8-serving-cert\""
Jul 28 01:47:05.879113 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:05.879089    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"]
Jul 28 01:47:05.986249 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:05.986210    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e36e728-24db-4f75-b8ec-49b0ca7b3150-maas-api-tls\") pod \"maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd\" (UID: \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:05.986249 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:05.986251    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bdpr\" (UniqueName: \"kubernetes.io/projected/9e36e728-24db-4f75-b8ec-49b0ca7b3150-kube-api-access-7bdpr\") pod \"maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd\" (UID: \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:05.986464 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:05.986322    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e36e728-24db-4f75-b8ec-49b0ca7b3150-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd\" (UID: \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:06.087655 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:06.087614    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e36e728-24db-4f75-b8ec-49b0ca7b3150-maas-api-tls\") pod \"maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd\" (UID: \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:06.087655 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:06.087657    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bdpr\" (UniqueName: \"kubernetes.io/projected/9e36e728-24db-4f75-b8ec-49b0ca7b3150-kube-api-access-7bdpr\") pod \"maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd\" (UID: \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:06.087895 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:06.087701    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e36e728-24db-4f75-b8ec-49b0ca7b3150-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd\" (UID: \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:06.087895 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:47:06.087767    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-serving-cert: secret "maas-api-e2e-shared-a-0337e8-serving-cert" not found
Jul 28 01:47:06.087895 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:47:06.087835    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e36e728-24db-4f75-b8ec-49b0ca7b3150-maas-api-tls podName:9e36e728-24db-4f75-b8ec-49b0ca7b3150 nodeName:}" failed. No retries permitted until 2026-07-28 01:47:06.587813831 +0000 UTC m=+2577.632707777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9e36e728-24db-4f75-b8ec-49b0ca7b3150-maas-api-tls") pod "maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd" (UID: "9e36e728-24db-4f75-b8ec-49b0ca7b3150") : secret "maas-api-e2e-shared-a-0337e8-serving-cert" not found
Jul 28 01:47:06.088360 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:06.088338    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e36e728-24db-4f75-b8ec-49b0ca7b3150-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd\" (UID: \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:06.099266 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:06.099236    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bdpr\" (UniqueName: \"kubernetes.io/projected/9e36e728-24db-4f75-b8ec-49b0ca7b3150-kube-api-access-7bdpr\") pod \"maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd\" (UID: \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:06.592106 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:06.592073    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e36e728-24db-4f75-b8ec-49b0ca7b3150-maas-api-tls\") pod \"maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd\" (UID: \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:06.592305 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:47:06.592208    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-serving-cert: secret "maas-api-e2e-shared-a-0337e8-serving-cert" not found
Jul 28 01:47:06.592305 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:47:06.592281    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e36e728-24db-4f75-b8ec-49b0ca7b3150-maas-api-tls podName:9e36e728-24db-4f75-b8ec-49b0ca7b3150 nodeName:}" failed. No retries permitted until 2026-07-28 01:47:07.592262297 +0000 UTC m=+2578.637156267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9e36e728-24db-4f75-b8ec-49b0ca7b3150-maas-api-tls") pod "maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd" (UID: "9e36e728-24db-4f75-b8ec-49b0ca7b3150") : secret "maas-api-e2e-shared-a-0337e8-serving-cert" not found
Jul 28 01:47:07.603807 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:07.603768    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e36e728-24db-4f75-b8ec-49b0ca7b3150-maas-api-tls\") pod \"maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd\" (UID: \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:07.609838 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:07.609792    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e36e728-24db-4f75-b8ec-49b0ca7b3150-maas-api-tls\") pod \"maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd\" (UID: \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:07.682678 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:07.682650    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:07.845898 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:07.845875    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"]
Jul 28 01:47:07.847947 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:47:07.847921    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e36e728_24db_4f75_b8ec_49b0ca7b3150.slice/crio-fbf2dd53040c264b100a2507e2c166850e4c5ffedaaf77fb554649ff30ad4cef WatchSource:0}: Error finding container fbf2dd53040c264b100a2507e2c166850e4c5ffedaaf77fb554649ff30ad4cef: Status 404 returned error can't find the container with id fbf2dd53040c264b100a2507e2c166850e4c5ffedaaf77fb554649ff30ad4cef
Jul 28 01:47:07.982667 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:07.982580    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd" event={"ID":"9e36e728-24db-4f75-b8ec-49b0ca7b3150","Type":"ContainerStarted","Data":"fbf2dd53040c264b100a2507e2c166850e4c5ffedaaf77fb554649ff30ad4cef"}
Jul 28 01:47:08.989249 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:08.989218    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd" event={"ID":"9e36e728-24db-4f75-b8ec-49b0ca7b3150","Type":"ContainerStarted","Data":"2367e6b360f2053e1ae4865f9e4a2d78b06f571518ccc1e86465013ac874fa4c"}
Jul 28 01:47:08.989623 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:08.989328    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:09.007220 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:09.007168    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd" podStartSLOduration=3.51412224 podStartE2EDuration="4.007154064s" totalImagesPullingTime="493.031824ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:47:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:47:09.00494856 +0000 UTC m=+2580.049842527" watchObservedRunningTime="2026-07-28 01:47:09.007154064 +0000 UTC m=+2580.052048057"
Jul 28 01:47:14.999808 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:14.999775    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 01:47:34.234685 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.234645    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"]
Jul 28 01:47:34.238380 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.238362    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 01:47:34.241300 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.241283    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-44b320-serving-cert\""
Jul 28 01:47:34.249603 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.249583    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"]
Jul 28 01:47:34.336190 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.336154    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27d96893-46fd-4476-994f-cb08c1f1b02f-maas-api-tls\") pod \"maas-api-e2e-shared-b-44b320-66475f898d-74x8w\" (UID: \"27d96893-46fd-4476-994f-cb08c1f1b02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 01:47:34.336360 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.336221    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x87zb\" (UniqueName: \"kubernetes.io/projected/27d96893-46fd-4476-994f-cb08c1f1b02f-kube-api-access-x87zb\") pod \"maas-api-e2e-shared-b-44b320-66475f898d-74x8w\" (UID: \"27d96893-46fd-4476-994f-cb08c1f1b02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 01:47:34.336360 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.336295    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d96893-46fd-4476-994f-cb08c1f1b02f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-44b320-66475f898d-74x8w\" (UID: \"27d96893-46fd-4476-994f-cb08c1f1b02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 01:47:34.437503 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.437455    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x87zb\" (UniqueName: \"kubernetes.io/projected/27d96893-46fd-4476-994f-cb08c1f1b02f-kube-api-access-x87zb\") pod \"maas-api-e2e-shared-b-44b320-66475f898d-74x8w\" (UID: \"27d96893-46fd-4476-994f-cb08c1f1b02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 01:47:34.437689 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.437551    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d96893-46fd-4476-994f-cb08c1f1b02f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-44b320-66475f898d-74x8w\" (UID: \"27d96893-46fd-4476-994f-cb08c1f1b02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 01:47:34.437689 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.437598    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27d96893-46fd-4476-994f-cb08c1f1b02f-maas-api-tls\") pod \"maas-api-e2e-shared-b-44b320-66475f898d-74x8w\" (UID: \"27d96893-46fd-4476-994f-cb08c1f1b02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 01:47:34.438336 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.438311    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d96893-46fd-4476-994f-cb08c1f1b02f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-44b320-66475f898d-74x8w\" (UID: \"27d96893-46fd-4476-994f-cb08c1f1b02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 01:47:34.440254 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.440231    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27d96893-46fd-4476-994f-cb08c1f1b02f-maas-api-tls\") pod \"maas-api-e2e-shared-b-44b320-66475f898d-74x8w\" (UID: \"27d96893-46fd-4476-994f-cb08c1f1b02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 01:47:34.456463 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.456433    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x87zb\" (UniqueName: \"kubernetes.io/projected/27d96893-46fd-4476-994f-cb08c1f1b02f-kube-api-access-x87zb\") pod \"maas-api-e2e-shared-b-44b320-66475f898d-74x8w\" (UID: \"27d96893-46fd-4476-994f-cb08c1f1b02f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 01:47:34.549570 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.549495    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 01:47:34.714711 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:34.714690    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"]
Jul 28 01:47:34.716241 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:47:34.716214    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d96893_46fd_4476_994f_cb08c1f1b02f.slice/crio-2f1bd698a6b07898ae90e758116f715ff16da43bd6d5e92c7effca3a4269ecdf WatchSource:0}: Error finding container 2f1bd698a6b07898ae90e758116f715ff16da43bd6d5e92c7effca3a4269ecdf: Status 404 returned error can't find the container with id 2f1bd698a6b07898ae90e758116f715ff16da43bd6d5e92c7effca3a4269ecdf
Jul 28 01:47:35.110035 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:35.110000    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w" event={"ID":"27d96893-46fd-4476-994f-cb08c1f1b02f","Type":"ContainerStarted","Data":"2f1bd698a6b07898ae90e758116f715ff16da43bd6d5e92c7effca3a4269ecdf"}
Jul 28 01:47:36.122856 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:36.122824    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w" event={"ID":"27d96893-46fd-4476-994f-cb08c1f1b02f","Type":"ContainerStarted","Data":"683e15c8f55f7cc003b26663333a500fa87dae6bbb91c8b0dfae7e6ee3686cc0"}
Jul 28 01:47:36.123276 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:36.122912    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 01:47:36.140975 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:36.140931    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w" podStartSLOduration=1.7344491579999999 podStartE2EDuration="2.140917424s" totalImagesPullingTime="406.468266ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:47:36.139770958 +0000 UTC m=+2607.184664936" watchObservedRunningTime="2026-07-28 01:47:36.140917424 +0000 UTC m=+2607.185811391"
Jul 28 01:47:42.133331 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:47:42.133299    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 01:48:25.528524 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.528470    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"]
Jul 28 01:48:25.532826 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.532804    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.535643 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.535621    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-44b320\"/\"kube-root-ca.crt\""
Jul 28 01:48:25.536692 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.536662    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-44b320\"/\"auth-test-model-44b320-kserve-self-signed-certs\""
Jul 28 01:48:25.536692 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.536687    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-44b320\"/\"openshift-service-ca.crt\""
Jul 28 01:48:25.536836 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.536813    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-44b320\"/\"default-dockercfg-qjqfd\""
Jul 28 01:48:25.542361 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.542339    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"]
Jul 28 01:48:25.600229 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.600191    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-home\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.600407 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.600233    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-model-cache\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.600475 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.600444    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-dshm\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.600659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.600478    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-tmp-dir\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.600659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.600541    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59s69\" (UniqueName: \"kubernetes.io/projected/737107f7-a43c-41dc-90d7-8f6fca9b0a69-kube-api-access-59s69\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.600659 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.600609    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/737107f7-a43c-41dc-90d7-8f6fca9b0a69-tls-certs\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.701523 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.701466    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-home\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.701523 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.701522    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-model-cache\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.701768 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.701612    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-dshm\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.701768 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.701638    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-tmp-dir\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.701768 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.701671    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59s69\" (UniqueName: \"kubernetes.io/projected/737107f7-a43c-41dc-90d7-8f6fca9b0a69-kube-api-access-59s69\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.701768 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.701705    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/737107f7-a43c-41dc-90d7-8f6fca9b0a69-tls-certs\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.701969 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.701942    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-home\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.702024 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.701968    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-model-cache\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.702074 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.702025    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-tmp-dir\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.704053 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.704028    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-dshm\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.704317 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.704301    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/737107f7-a43c-41dc-90d7-8f6fca9b0a69-tls-certs\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.710805 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.710781    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59s69\" (UniqueName: \"kubernetes.io/projected/737107f7-a43c-41dc-90d7-8f6fca9b0a69-kube-api-access-59s69\") pod \"auth-test-model-44b320-kserve-c57bb6b99-xcr8b\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") " pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.843653 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.843621    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:25.975921 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:25.975895    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"]
Jul 28 01:48:25.978061 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:48:25.978021    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod737107f7_a43c_41dc_90d7_8f6fca9b0a69.slice/crio-92fca62b6414562aad7a3d748f74162f0a48994c66e6eee39f58c2efa46be37f WatchSource:0}: Error finding container 92fca62b6414562aad7a3d748f74162f0a48994c66e6eee39f58c2efa46be37f: Status 404 returned error can't find the container with id 92fca62b6414562aad7a3d748f74162f0a48994c66e6eee39f58c2efa46be37f
Jul 28 01:48:26.356776 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:26.356737    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b" event={"ID":"737107f7-a43c-41dc-90d7-8f6fca9b0a69","Type":"ContainerStarted","Data":"9eb454c83db9ed6b5fda57274882f96af0743ef18bddcf6abd7e0a3012bf7ffc"}
Jul 28 01:48:26.356776 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:26.356783    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b" event={"ID":"737107f7-a43c-41dc-90d7-8f6fca9b0a69","Type":"ContainerStarted","Data":"92fca62b6414562aad7a3d748f74162f0a48994c66e6eee39f58c2efa46be37f"}
Jul 28 01:48:26.379869 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:26.379822    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b" podStartSLOduration=1.379803095 podStartE2EDuration="1.379803095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:48:26.379219872 +0000 UTC m=+2657.424113878" watchObservedRunningTime="2026-07-28 01:48:26.379803095 +0000 UTC m=+2657.424697062"
Jul 28 01:48:35.844367 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:35.844329    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:35.844795 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:35.844424    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:35.862766 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:35.862740    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:48:36.416595 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:48:36.416555    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:49:10.256783 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:49:10.256677    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:49:10.258249 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:49:10.257637    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:49:10.263385 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:49:10.263366    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:49:10.263780 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:49:10.263763    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:49:10.264255 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:49:10.264231    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:49:10.270571 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:49:10.270555    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:52:53.005736 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:52:53.005705    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"]
Jul 28 01:53:08.025365 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.025317    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b" podUID="737107f7-a43c-41dc-90d7-8f6fca9b0a69" containerName="main" containerID="cri-o://9eb454c83db9ed6b5fda57274882f96af0743ef18bddcf6abd7e0a3012bf7ffc" gracePeriod=45
Jul 28 01:53:08.273207 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.273182    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:53:08.319181 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.319147    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/737107f7-a43c-41dc-90d7-8f6fca9b0a69-tls-certs\") pod \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") "
Jul 28 01:53:08.319359 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.319211    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59s69\" (UniqueName: \"kubernetes.io/projected/737107f7-a43c-41dc-90d7-8f6fca9b0a69-kube-api-access-59s69\") pod \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") "
Jul 28 01:53:08.319359 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.319287    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-model-cache\") pod \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") "
Jul 28 01:53:08.319359 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.319312    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-tmp-dir\") pod \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") "
Jul 28 01:53:08.319359 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.319338    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-home\") pod \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") "
Jul 28 01:53:08.319650 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.319411    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-dshm\") pod \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\" (UID: \"737107f7-a43c-41dc-90d7-8f6fca9b0a69\") "
Jul 28 01:53:08.319718 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.319657    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-model-cache" (OuterVolumeSpecName: "model-cache") pod "737107f7-a43c-41dc-90d7-8f6fca9b0a69" (UID: "737107f7-a43c-41dc-90d7-8f6fca9b0a69"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:53:08.319778 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.319743    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "737107f7-a43c-41dc-90d7-8f6fca9b0a69" (UID: "737107f7-a43c-41dc-90d7-8f6fca9b0a69"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:53:08.319832 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.319765    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-home" (OuterVolumeSpecName: "home") pod "737107f7-a43c-41dc-90d7-8f6fca9b0a69" (UID: "737107f7-a43c-41dc-90d7-8f6fca9b0a69"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:53:08.321705 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.321676    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/737107f7-a43c-41dc-90d7-8f6fca9b0a69-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "737107f7-a43c-41dc-90d7-8f6fca9b0a69" (UID: "737107f7-a43c-41dc-90d7-8f6fca9b0a69"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:53:08.321837 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.321821    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-dshm" (OuterVolumeSpecName: "dshm") pod "737107f7-a43c-41dc-90d7-8f6fca9b0a69" (UID: "737107f7-a43c-41dc-90d7-8f6fca9b0a69"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:53:08.321889 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.321858    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/737107f7-a43c-41dc-90d7-8f6fca9b0a69-kube-api-access-59s69" (OuterVolumeSpecName: "kube-api-access-59s69") pod "737107f7-a43c-41dc-90d7-8f6fca9b0a69" (UID: "737107f7-a43c-41dc-90d7-8f6fca9b0a69"). InnerVolumeSpecName "kube-api-access-59s69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:53:08.420831 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.420801    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59s69\" (UniqueName: \"kubernetes.io/projected/737107f7-a43c-41dc-90d7-8f6fca9b0a69-kube-api-access-59s69\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:53:08.420831 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.420828    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-model-cache\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:53:08.421012 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.420856    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-tmp-dir\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:53:08.421012 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.420864    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-home\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:53:08.421012 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.420871    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/737107f7-a43c-41dc-90d7-8f6fca9b0a69-dshm\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:53:08.421012 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.420877    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/737107f7-a43c-41dc-90d7-8f6fca9b0a69-tls-certs\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:53:08.697047 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.696945    2565 generic.go:356] "Generic (PLEG): container finished" podID="737107f7-a43c-41dc-90d7-8f6fca9b0a69" containerID="9eb454c83db9ed6b5fda57274882f96af0743ef18bddcf6abd7e0a3012bf7ffc" exitCode=0
Jul 28 01:53:08.697047 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.696990    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"
Jul 28 01:53:08.697047 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.697023    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b" event={"ID":"737107f7-a43c-41dc-90d7-8f6fca9b0a69","Type":"ContainerDied","Data":"9eb454c83db9ed6b5fda57274882f96af0743ef18bddcf6abd7e0a3012bf7ffc"}
Jul 28 01:53:08.697298 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.697063    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b" event={"ID":"737107f7-a43c-41dc-90d7-8f6fca9b0a69","Type":"ContainerDied","Data":"92fca62b6414562aad7a3d748f74162f0a48994c66e6eee39f58c2efa46be37f"}
Jul 28 01:53:08.697298 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.697083    2565 scope.go:117] "RemoveContainer" containerID="9eb454c83db9ed6b5fda57274882f96af0743ef18bddcf6abd7e0a3012bf7ffc"
Jul 28 01:53:08.706637 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.706619    2565 scope.go:117] "RemoveContainer" containerID="9eb454c83db9ed6b5fda57274882f96af0743ef18bddcf6abd7e0a3012bf7ffc"
Jul 28 01:53:08.706857 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:53:08.706842    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9eb454c83db9ed6b5fda57274882f96af0743ef18bddcf6abd7e0a3012bf7ffc\": container with ID starting with 9eb454c83db9ed6b5fda57274882f96af0743ef18bddcf6abd7e0a3012bf7ffc not found: ID does not exist" containerID="9eb454c83db9ed6b5fda57274882f96af0743ef18bddcf6abd7e0a3012bf7ffc"
Jul 28 01:53:08.706909 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.706865    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eb454c83db9ed6b5fda57274882f96af0743ef18bddcf6abd7e0a3012bf7ffc"} err="failed to get container status \"9eb454c83db9ed6b5fda57274882f96af0743ef18bddcf6abd7e0a3012bf7ffc\": rpc error: code = NotFound desc = could not find container \"9eb454c83db9ed6b5fda57274882f96af0743ef18bddcf6abd7e0a3012bf7ffc\": container with ID starting with 9eb454c83db9ed6b5fda57274882f96af0743ef18bddcf6abd7e0a3012bf7ffc not found: ID does not exist"
Jul 28 01:53:08.722079 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.722049    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"]
Jul 28 01:53:08.723704 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:08.723684    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/auth-test-model-44b320-kserve-c57bb6b99-xcr8b"]
Jul 28 01:53:09.467266 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:53:09.467231    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="737107f7-a43c-41dc-90d7-8f6fca9b0a69" path="/var/lib/kubelet/pods/737107f7-a43c-41dc-90d7-8f6fca9b0a69/volumes"
Jul 28 01:54:10.288247 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:54:10.288134    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:54:10.289347 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:54:10.288933    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:54:10.294929 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:54:10.294907    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:54:10.295457 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:54:10.295440    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:54:10.296528 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:54:10.296511    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:54:10.302168 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:54:10.302154    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:55:31.431940 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.431906    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"]
Jul 28 01:55:31.432470 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.432454    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="737107f7-a43c-41dc-90d7-8f6fca9b0a69" containerName="main"
Jul 28 01:55:31.432532 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.432473    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="737107f7-a43c-41dc-90d7-8f6fca9b0a69" containerName="main"
Jul 28 01:55:31.432611 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.432600    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="737107f7-a43c-41dc-90d7-8f6fca9b0a69" containerName="main"
Jul 28 01:55:31.435917 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.435896    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.440145 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.440125    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-44b320\"/\"rate-test-model-44b320-kserve-self-signed-certs\""
Jul 28 01:55:31.440145 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.440143    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-44b320\"/\"kube-root-ca.crt\""
Jul 28 01:55:31.440331 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.440126    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-44b320\"/\"default-dockercfg-qjqfd\""
Jul 28 01:55:31.440331 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.440126    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-44b320\"/\"openshift-service-ca.crt\""
Jul 28 01:55:31.444648 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.444622    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"]
Jul 28 01:55:31.507899 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.507867    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82d00d2b-b76d-457f-99f0-e7368efc3b72-tls-certs\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.507899 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.507902    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-model-cache\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.508144 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.507920    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-dshm\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.508144 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.507952    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-tmp-dir\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.508144 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.508115    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfpc2\" (UniqueName: \"kubernetes.io/projected/82d00d2b-b76d-457f-99f0-e7368efc3b72-kube-api-access-xfpc2\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.508312 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.508147    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-home\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.609021 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.608984    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-tmp-dir\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.609181 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.609049    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfpc2\" (UniqueName: \"kubernetes.io/projected/82d00d2b-b76d-457f-99f0-e7368efc3b72-kube-api-access-xfpc2\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.609181 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.609076    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-home\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.609181 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.609118    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82d00d2b-b76d-457f-99f0-e7368efc3b72-tls-certs\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.609181 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.609140    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-model-cache\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.609319 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.609266    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-dshm\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.609441 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.609419    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-tmp-dir\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.609548 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.609444    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-model-cache\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.609548 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.609517    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-home\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.611593 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.611572    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-dshm\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.611720 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.611706    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82d00d2b-b76d-457f-99f0-e7368efc3b72-tls-certs\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.619165 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.619137    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfpc2\" (UniqueName: \"kubernetes.io/projected/82d00d2b-b76d-457f-99f0-e7368efc3b72-kube-api-access-xfpc2\") pod \"rate-test-model-44b320-kserve-64cbb776f9-8pxm2\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") " pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.745653 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.745560    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:31.884083 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:31.884052    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"]
Jul 28 01:55:31.888699 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:55:31.888671    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82d00d2b_b76d_457f_99f0_e7368efc3b72.slice/crio-31d4f64cff68a44f0139e30ab1431a59981b1bf88c7dc3a53fab3d25d93e14e4 WatchSource:0}: Error finding container 31d4f64cff68a44f0139e30ab1431a59981b1bf88c7dc3a53fab3d25d93e14e4: Status 404 returned error can't find the container with id 31d4f64cff68a44f0139e30ab1431a59981b1bf88c7dc3a53fab3d25d93e14e4
Jul 28 01:55:32.357404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:32.357366    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2" event={"ID":"82d00d2b-b76d-457f-99f0-e7368efc3b72","Type":"ContainerStarted","Data":"c179ede42a6480a64836d6d011ad6960e76cb230ab6f8fda7dc86d7aa605feda"}
Jul 28 01:55:32.357627 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:32.357411    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2" event={"ID":"82d00d2b-b76d-457f-99f0-e7368efc3b72","Type":"ContainerStarted","Data":"31d4f64cff68a44f0139e30ab1431a59981b1bf88c7dc3a53fab3d25d93e14e4"}
Jul 28 01:55:32.375202 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:32.375155    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2" podStartSLOduration=1.37514018 podStartE2EDuration="1.37514018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:55:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:55:32.3733048 +0000 UTC m=+3083.418198767" watchObservedRunningTime="2026-07-28 01:55:32.37514018 +0000 UTC m=+3083.420034147"
Jul 28 01:55:41.745863 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:41.745826    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:41.746326 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:41.746079    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:41.761531 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:41.761501    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:55:42.414018 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:55:42.413980    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:56:39.922942 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:39.922866    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"]
Jul 28 01:56:54.942557 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:54.942515    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2" podUID="82d00d2b-b76d-457f-99f0-e7368efc3b72" containerName="main" containerID="cri-o://c179ede42a6480a64836d6d011ad6960e76cb230ab6f8fda7dc86d7aa605feda" gracePeriod=45
Jul 28 01:56:55.198565 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.198477    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:56:55.300650 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.300617    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfpc2\" (UniqueName: \"kubernetes.io/projected/82d00d2b-b76d-457f-99f0-e7368efc3b72-kube-api-access-xfpc2\") pod \"82d00d2b-b76d-457f-99f0-e7368efc3b72\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") "
Jul 28 01:56:55.300804 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.300676    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-home\") pod \"82d00d2b-b76d-457f-99f0-e7368efc3b72\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") "
Jul 28 01:56:55.300804 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.300710    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-model-cache\") pod \"82d00d2b-b76d-457f-99f0-e7368efc3b72\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") "
Jul 28 01:56:55.300804 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.300729    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-tmp-dir\") pod \"82d00d2b-b76d-457f-99f0-e7368efc3b72\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") "
Jul 28 01:56:55.300804 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.300774    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82d00d2b-b76d-457f-99f0-e7368efc3b72-tls-certs\") pod \"82d00d2b-b76d-457f-99f0-e7368efc3b72\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") "
Jul 28 01:56:55.300999 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.300854    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-dshm\") pod \"82d00d2b-b76d-457f-99f0-e7368efc3b72\" (UID: \"82d00d2b-b76d-457f-99f0-e7368efc3b72\") "
Jul 28 01:56:55.301058 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.300998    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-model-cache" (OuterVolumeSpecName: "model-cache") pod "82d00d2b-b76d-457f-99f0-e7368efc3b72" (UID: "82d00d2b-b76d-457f-99f0-e7368efc3b72"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:56:55.301058 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.301005    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-home" (OuterVolumeSpecName: "home") pod "82d00d2b-b76d-457f-99f0-e7368efc3b72" (UID: "82d00d2b-b76d-457f-99f0-e7368efc3b72"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:56:55.301058 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.301028    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "82d00d2b-b76d-457f-99f0-e7368efc3b72" (UID: "82d00d2b-b76d-457f-99f0-e7368efc3b72"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:56:55.301198 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.301182    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-home\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:56:55.301234 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.301200    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-model-cache\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:56:55.301234 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.301212    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-tmp-dir\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:56:55.302894 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.302868    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82d00d2b-b76d-457f-99f0-e7368efc3b72-kube-api-access-xfpc2" (OuterVolumeSpecName: "kube-api-access-xfpc2") pod "82d00d2b-b76d-457f-99f0-e7368efc3b72" (UID: "82d00d2b-b76d-457f-99f0-e7368efc3b72"). InnerVolumeSpecName "kube-api-access-xfpc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:56:55.303044 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.303025    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-dshm" (OuterVolumeSpecName: "dshm") pod "82d00d2b-b76d-457f-99f0-e7368efc3b72" (UID: "82d00d2b-b76d-457f-99f0-e7368efc3b72"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:56:55.303105 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.303075    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82d00d2b-b76d-457f-99f0-e7368efc3b72-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "82d00d2b-b76d-457f-99f0-e7368efc3b72" (UID: "82d00d2b-b76d-457f-99f0-e7368efc3b72"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:56:55.401663 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.401630    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82d00d2b-b76d-457f-99f0-e7368efc3b72-tls-certs\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:56:55.401663 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.401657    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82d00d2b-b76d-457f-99f0-e7368efc3b72-dshm\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:56:55.401819 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.401669    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfpc2\" (UniqueName: \"kubernetes.io/projected/82d00d2b-b76d-457f-99f0-e7368efc3b72-kube-api-access-xfpc2\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:56:55.752216 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.752129    2565 generic.go:356] "Generic (PLEG): container finished" podID="82d00d2b-b76d-457f-99f0-e7368efc3b72" containerID="c179ede42a6480a64836d6d011ad6960e76cb230ab6f8fda7dc86d7aa605feda" exitCode=0
Jul 28 01:56:55.752216 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.752170    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"
Jul 28 01:56:55.752216 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.752186    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2" event={"ID":"82d00d2b-b76d-457f-99f0-e7368efc3b72","Type":"ContainerDied","Data":"c179ede42a6480a64836d6d011ad6960e76cb230ab6f8fda7dc86d7aa605feda"}
Jul 28 01:56:55.752216 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.752217    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2" event={"ID":"82d00d2b-b76d-457f-99f0-e7368efc3b72","Type":"ContainerDied","Data":"31d4f64cff68a44f0139e30ab1431a59981b1bf88c7dc3a53fab3d25d93e14e4"}
Jul 28 01:56:55.752538 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.752233    2565 scope.go:117] "RemoveContainer" containerID="c179ede42a6480a64836d6d011ad6960e76cb230ab6f8fda7dc86d7aa605feda"
Jul 28 01:56:55.761169 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.761154    2565 scope.go:117] "RemoveContainer" containerID="c179ede42a6480a64836d6d011ad6960e76cb230ab6f8fda7dc86d7aa605feda"
Jul 28 01:56:55.761452 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:56:55.761435    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c179ede42a6480a64836d6d011ad6960e76cb230ab6f8fda7dc86d7aa605feda\": container with ID starting with c179ede42a6480a64836d6d011ad6960e76cb230ab6f8fda7dc86d7aa605feda not found: ID does not exist" containerID="c179ede42a6480a64836d6d011ad6960e76cb230ab6f8fda7dc86d7aa605feda"
Jul 28 01:56:55.761542 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.761462    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c179ede42a6480a64836d6d011ad6960e76cb230ab6f8fda7dc86d7aa605feda"} err="failed to get container status \"c179ede42a6480a64836d6d011ad6960e76cb230ab6f8fda7dc86d7aa605feda\": rpc error: code = NotFound desc = could not find container \"c179ede42a6480a64836d6d011ad6960e76cb230ab6f8fda7dc86d7aa605feda\": container with ID starting with c179ede42a6480a64836d6d011ad6960e76cb230ab6f8fda7dc86d7aa605feda not found: ID does not exist"
Jul 28 01:56:55.774666 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.774644    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"]
Jul 28 01:56:55.780189 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:55.780169    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/rate-test-model-44b320-kserve-64cbb776f9-8pxm2"]
Jul 28 01:56:57.466835 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:56:57.466799    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82d00d2b-b76d-457f-99f0-e7368efc3b72" path="/var/lib/kubelet/pods/82d00d2b-b76d-457f-99f0-e7368efc3b72/volumes"
Jul 28 01:57:02.365507 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.365441    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"]
Jul 28 01:57:02.366010 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.365996    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82d00d2b-b76d-457f-99f0-e7368efc3b72" containerName="main"
Jul 28 01:57:02.366306 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.366014    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="82d00d2b-b76d-457f-99f0-e7368efc3b72" containerName="main"
Jul 28 01:57:02.366306 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.366128    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="82d00d2b-b76d-457f-99f0-e7368efc3b72" containerName="main"
Jul 28 01:57:02.370957 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.370925    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:57:02.374533 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.374514    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-8a33ed-serving-cert\""
Jul 28 01:57:02.385890 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.385829    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"]
Jul 28 01:57:02.466908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.466830    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efffbb7c-c61c-4852-8657-78c64c9b0eeb-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv\" (UID: \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:57:02.466908 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.466873    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efffbb7c-c61c-4852-8657-78c64c9b0eeb-maas-api-tls\") pod \"maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv\" (UID: \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:57:02.467074 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.467049    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf6dq\" (UniqueName: \"kubernetes.io/projected/efffbb7c-c61c-4852-8657-78c64c9b0eeb-kube-api-access-sf6dq\") pod \"maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv\" (UID: \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:57:02.568096 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.568066    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf6dq\" (UniqueName: \"kubernetes.io/projected/efffbb7c-c61c-4852-8657-78c64c9b0eeb-kube-api-access-sf6dq\") pod \"maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv\" (UID: \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:57:02.568251 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.568113    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efffbb7c-c61c-4852-8657-78c64c9b0eeb-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv\" (UID: \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:57:02.568295 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.568237    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efffbb7c-c61c-4852-8657-78c64c9b0eeb-maas-api-tls\") pod \"maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv\" (UID: \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:57:02.568405 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:57:02.568384    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-serving-cert: secret "maas-api-e2e-ipp-a-8a33ed-serving-cert" not found
Jul 28 01:57:02.568506 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:57:02.568452    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efffbb7c-c61c-4852-8657-78c64c9b0eeb-maas-api-tls podName:efffbb7c-c61c-4852-8657-78c64c9b0eeb nodeName:}" failed. No retries permitted until 2026-07-28 01:57:03.068431637 +0000 UTC m=+3174.113325602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/efffbb7c-c61c-4852-8657-78c64c9b0eeb-maas-api-tls") pod "maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv" (UID: "efffbb7c-c61c-4852-8657-78c64c9b0eeb") : secret "maas-api-e2e-ipp-a-8a33ed-serving-cert" not found
Jul 28 01:57:02.568876 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.568854    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efffbb7c-c61c-4852-8657-78c64c9b0eeb-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv\" (UID: \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:57:02.580653 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:02.580624    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf6dq\" (UniqueName: \"kubernetes.io/projected/efffbb7c-c61c-4852-8657-78c64c9b0eeb-kube-api-access-sf6dq\") pod \"maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv\" (UID: \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:57:03.072290 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:03.072259    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efffbb7c-c61c-4852-8657-78c64c9b0eeb-maas-api-tls\") pod \"maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv\" (UID: \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:57:03.074972 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:03.074947    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efffbb7c-c61c-4852-8657-78c64c9b0eeb-maas-api-tls\") pod \"maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv\" (UID: \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:57:03.281554 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:03.281524    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:57:03.421207 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:03.421058    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"]
Jul 28 01:57:03.423948 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:57:03.423915    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefffbb7c_c61c_4852_8657_78c64c9b0eeb.slice/crio-92fa6f91f5a455c848b8ee6a35edcb2deffbdb39bef1bf94adf6f82ef74e45b8 WatchSource:0}: Error finding container 92fa6f91f5a455c848b8ee6a35edcb2deffbdb39bef1bf94adf6f82ef74e45b8: Status 404 returned error can't find the container with id 92fa6f91f5a455c848b8ee6a35edcb2deffbdb39bef1bf94adf6f82ef74e45b8
Jul 28 01:57:03.425397 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:03.425379    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:57:03.796145 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:03.796063    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv" event={"ID":"efffbb7c-c61c-4852-8657-78c64c9b0eeb","Type":"ContainerStarted","Data":"92fa6f91f5a455c848b8ee6a35edcb2deffbdb39bef1bf94adf6f82ef74e45b8"}
Jul 28 01:57:04.802626 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:04.802579    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv" event={"ID":"efffbb7c-c61c-4852-8657-78c64c9b0eeb","Type":"ContainerStarted","Data":"f8d800421a56541bd96db895eb7eac942b1274aabc90932ec9c6c9fad12a1459"}
Jul 28 01:57:04.802997 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:04.802714    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:57:04.829456 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:04.829414    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv" podStartSLOduration=2.346093561 podStartE2EDuration="2.829398289s" totalImagesPullingTime="483.304728ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:57:04.828453644 +0000 UTC m=+3175.873347610" watchObservedRunningTime="2026-07-28 01:57:04.829398289 +0000 UTC m=+3175.874292255"
Jul 28 01:57:10.812061 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:10.812032    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:57:31.818906 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:31.818867    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"]
Jul 28 01:57:31.822924 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:31.822905    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 01:57:31.825600 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:31.825579    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-c430e4-serving-cert\""
Jul 28 01:57:31.836582 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:31.836560    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"]
Jul 28 01:57:31.934228 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:31.934198    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhx9b\" (UniqueName: \"kubernetes.io/projected/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-kube-api-access-zhx9b\") pod \"maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx\" (UID: \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 01:57:31.934373 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:31.934232    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx\" (UID: \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 01:57:31.934373 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:31.934298    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx\" (UID: \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 01:57:32.035733 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:32.035702    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx\" (UID: \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 01:57:32.035890 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:32.035790    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhx9b\" (UniqueName: \"kubernetes.io/projected/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-kube-api-access-zhx9b\") pod \"maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx\" (UID: \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 01:57:32.035890 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:32.035810    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx\" (UID: \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 01:57:32.035890 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:57:32.035875    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-serving-cert: secret "maas-api-e2e-ipp-b-c430e4-serving-cert" not found
Jul 28 01:57:32.036055 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:57:32.035963    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-maas-api-tls podName:0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23 nodeName:}" failed. No retries permitted until 2026-07-28 01:57:32.535939052 +0000 UTC m=+3203.580833005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-maas-api-tls") pod "maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx" (UID: "0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23") : secret "maas-api-e2e-ipp-b-c430e4-serving-cert" not found
Jul 28 01:57:32.036389 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:32.036374    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx\" (UID: \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 01:57:32.044952 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:32.044932    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhx9b\" (UniqueName: \"kubernetes.io/projected/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-kube-api-access-zhx9b\") pod \"maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx\" (UID: \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 01:57:32.540211 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:32.540182    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx\" (UID: \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 01:57:32.542705 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:32.542676    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx\" (UID: \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 01:57:32.734991 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:32.734960    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 01:57:32.869314 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:32.869289    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"]
Jul 28 01:57:32.871639 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:57:32.871611    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0901efc1_9ce6_46f8_8fb4_f5e3f73e3f23.slice/crio-9f9a8e398bfa45f369ab22c1941942a2045a81559872e96814e8e3e03a40a051 WatchSource:0}: Error finding container 9f9a8e398bfa45f369ab22c1941942a2045a81559872e96814e8e3e03a40a051: Status 404 returned error can't find the container with id 9f9a8e398bfa45f369ab22c1941942a2045a81559872e96814e8e3e03a40a051
Jul 28 01:57:32.949955 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:32.949923    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx" event={"ID":"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23","Type":"ContainerStarted","Data":"9f9a8e398bfa45f369ab22c1941942a2045a81559872e96814e8e3e03a40a051"}
Jul 28 01:57:33.956106 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:33.956075    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx" event={"ID":"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23","Type":"ContainerStarted","Data":"08197d6e0a0f4e5ad985d6581771610889e9569a450bf8c7ecc6b87cfc294fb5"}
Jul 28 01:57:33.956478 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:33.956135    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 01:57:33.975387 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:33.975333    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx" podStartSLOduration=2.543747471 podStartE2EDuration="2.975318482s" totalImagesPullingTime="431.571011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:57:33.97374302 +0000 UTC m=+3205.018636988" watchObservedRunningTime="2026-07-28 01:57:33.975318482 +0000 UTC m=+3205.020212481"
Jul 28 01:57:39.967077 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:57:39.967046    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 01:58:02.222849 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.222813    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"]
Jul 28 01:58:02.226124 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.226104    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.230026 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.230004    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-8a33ed\"/\"ipp-route-8a33ed-kserve-self-signed-certs\""
Jul 28 01:58:02.230154 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.230101    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-8a33ed\"/\"default-dockercfg-4cmp8\""
Jul 28 01:58:02.230219 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.230157    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-8a33ed\"/\"openshift-service-ca.crt\""
Jul 28 01:58:02.230219 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.230196    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-8a33ed\"/\"kube-root-ca.crt\""
Jul 28 01:58:02.238009 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.237988    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"]
Jul 28 01:58:02.322678 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.322642    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-home\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.322874 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.322687    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-dshm\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.322874 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.322776    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-tmp-dir\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.322874 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.322866    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcppr\" (UniqueName: \"kubernetes.io/projected/f8f6dd2b-f49b-4268-8241-4d333e36962c-kube-api-access-xcppr\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.323044 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.322897    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8f6dd2b-f49b-4268-8241-4d333e36962c-tls-certs\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.323044 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.322923    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-model-cache\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.423798 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.423765    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-model-cache\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.423995 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.423816    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-home\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.423995 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.423844    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-dshm\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.423995 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.423881    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-tmp-dir\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.423995 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.423966    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcppr\" (UniqueName: \"kubernetes.io/projected/f8f6dd2b-f49b-4268-8241-4d333e36962c-kube-api-access-xcppr\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.423995 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.423993    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8f6dd2b-f49b-4268-8241-4d333e36962c-tls-certs\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.424307 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.424282    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-model-cache\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.424373 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.424312    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-home\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.424458 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.424438    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-tmp-dir\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.426620 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.426586    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-dshm\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.426817 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.426802    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8f6dd2b-f49b-4268-8241-4d333e36962c-tls-certs\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.435805 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.435693    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcppr\" (UniqueName: \"kubernetes.io/projected/f8f6dd2b-f49b-4268-8241-4d333e36962c-kube-api-access-xcppr\") pod \"ipp-route-8a33ed-kserve-846679f5bc-spbqf\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") " pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.536762 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.536721    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:02.673041 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:02.673018    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"]
Jul 28 01:58:02.674902 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:58:02.674874    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8f6dd2b_f49b_4268_8241_4d333e36962c.slice/crio-1c633de7ca30b810c6f2ec63f509476662d84cda547428fd9053394dfd4dcc04 WatchSource:0}: Error finding container 1c633de7ca30b810c6f2ec63f509476662d84cda547428fd9053394dfd4dcc04: Status 404 returned error can't find the container with id 1c633de7ca30b810c6f2ec63f509476662d84cda547428fd9053394dfd4dcc04
Jul 28 01:58:03.096330 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:03.096298    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf" event={"ID":"f8f6dd2b-f49b-4268-8241-4d333e36962c","Type":"ContainerStarted","Data":"f6d27f93063fba09346242118af8e829cb23ba398895c7fe596bcec8751c5e85"}
Jul 28 01:58:03.096330 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:03.096344    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf" event={"ID":"f8f6dd2b-f49b-4268-8241-4d333e36962c","Type":"ContainerStarted","Data":"1c633de7ca30b810c6f2ec63f509476662d84cda547428fd9053394dfd4dcc04"}
Jul 28 01:58:03.117059 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:03.117008    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf" podStartSLOduration=1.116993576 podStartE2EDuration="1.116993576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:58:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:58:03.115374864 +0000 UTC m=+3234.160268832" watchObservedRunningTime="2026-07-28 01:58:03.116993576 +0000 UTC m=+3234.161887540"
Jul 28 01:58:12.537324 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:12.537287    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:12.537942 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:12.537347    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:12.549674 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:12.549651    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:13.156151 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:13.156119    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:58:49.013163 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.013123    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"]
Jul 28 01:58:49.016750 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.016732    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:58:49.019455 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.019431    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-9e1b27-serving-cert\""
Jul 28 01:58:49.028421 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.028385    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"]
Jul 28 01:58:49.066019 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.065982    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51adf7ef-20cd-402a-bb20-6804f7b8f891-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x\" (UID: \"51adf7ef-20cd-402a-bb20-6804f7b8f891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:58:49.066019 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.066024    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51adf7ef-20cd-402a-bb20-6804f7b8f891-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x\" (UID: \"51adf7ef-20cd-402a-bb20-6804f7b8f891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:58:49.066259 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.066160    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5pfv\" (UniqueName: \"kubernetes.io/projected/51adf7ef-20cd-402a-bb20-6804f7b8f891-kube-api-access-p5pfv\") pod \"maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x\" (UID: \"51adf7ef-20cd-402a-bb20-6804f7b8f891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:58:49.167155 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.167117    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5pfv\" (UniqueName: \"kubernetes.io/projected/51adf7ef-20cd-402a-bb20-6804f7b8f891-kube-api-access-p5pfv\") pod \"maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x\" (UID: \"51adf7ef-20cd-402a-bb20-6804f7b8f891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:58:49.167338 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.167219    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51adf7ef-20cd-402a-bb20-6804f7b8f891-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x\" (UID: \"51adf7ef-20cd-402a-bb20-6804f7b8f891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:58:49.167338 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.167249    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51adf7ef-20cd-402a-bb20-6804f7b8f891-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x\" (UID: \"51adf7ef-20cd-402a-bb20-6804f7b8f891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:58:49.167426 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:58:49.167366    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-serving-cert: secret "maas-api-e2e-ipp-cleanup-9e1b27-serving-cert" not found
Jul 28 01:58:49.167464 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:58:49.167446    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51adf7ef-20cd-402a-bb20-6804f7b8f891-maas-api-tls podName:51adf7ef-20cd-402a-bb20-6804f7b8f891 nodeName:}" failed. No retries permitted until 2026-07-28 01:58:49.667426878 +0000 UTC m=+3280.712320841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/51adf7ef-20cd-402a-bb20-6804f7b8f891-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x" (UID: "51adf7ef-20cd-402a-bb20-6804f7b8f891") : secret "maas-api-e2e-ipp-cleanup-9e1b27-serving-cert" not found
Jul 28 01:58:49.167964 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.167945    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51adf7ef-20cd-402a-bb20-6804f7b8f891-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x\" (UID: \"51adf7ef-20cd-402a-bb20-6804f7b8f891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:58:49.176641 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.176614    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5pfv\" (UniqueName: \"kubernetes.io/projected/51adf7ef-20cd-402a-bb20-6804f7b8f891-kube-api-access-p5pfv\") pod \"maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x\" (UID: \"51adf7ef-20cd-402a-bb20-6804f7b8f891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:58:49.674553 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.674477    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51adf7ef-20cd-402a-bb20-6804f7b8f891-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x\" (UID: \"51adf7ef-20cd-402a-bb20-6804f7b8f891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:58:49.677195 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.677166    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51adf7ef-20cd-402a-bb20-6804f7b8f891-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x\" (UID: \"51adf7ef-20cd-402a-bb20-6804f7b8f891\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:58:49.929307 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:49.929216    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:58:50.074019 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:50.073992    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"]
Jul 28 01:58:50.075986 ip-10-0-138-173 kubenswrapper[2565]: W0728 01:58:50.075957    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51adf7ef_20cd_402a_bb20_6804f7b8f891.slice/crio-8017cd422377a172b0ada751b51bc6fb6e87640fdc20bde98f48c38d34751a7f WatchSource:0}: Error finding container 8017cd422377a172b0ada751b51bc6fb6e87640fdc20bde98f48c38d34751a7f: Status 404 returned error can't find the container with id 8017cd422377a172b0ada751b51bc6fb6e87640fdc20bde98f48c38d34751a7f
Jul 28 01:58:50.320750 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:50.320716    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x" event={"ID":"51adf7ef-20cd-402a-bb20-6804f7b8f891","Type":"ContainerStarted","Data":"8017cd422377a172b0ada751b51bc6fb6e87640fdc20bde98f48c38d34751a7f"}
Jul 28 01:58:51.327269 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:51.327224    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x" event={"ID":"51adf7ef-20cd-402a-bb20-6804f7b8f891","Type":"ContainerStarted","Data":"74c9bc83088601425e1c3225e6b54de0e0daa9ba67e24f0de5adf67dbbaf393f"}
Jul 28 01:58:51.327676 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:51.327286    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:58:57.337404 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:57.337378    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:58:57.355733 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:58:57.355692    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x" podStartSLOduration=8.96121151 podStartE2EDuration="9.355677266s" totalImagesPullingTime="394.465756ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:58:51.345201751 +0000 UTC m=+3282.390095719" watchObservedRunningTime="2026-07-28 01:58:57.355677266 +0000 UTC m=+3288.400571263"
Jul 28 01:59:10.318302 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:10.318279    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:59:10.320711 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:10.319336    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:59:10.325336 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:10.325320    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:59:10.326617 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:10.326602    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 01:59:10.327467 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:10.327452    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 01:59:10.333153 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:10.333137    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5qk5p_df8d0717-00ff-43f0-9b4e-7733475844b2/ovn-acl-logging/0.log"
Jul 28 01:59:11.272343 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:11.272311    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"]
Jul 28 01:59:11.272609 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:11.272562    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x" podUID="51adf7ef-20cd-402a-bb20-6804f7b8f891" containerName="maas-api" containerID="cri-o://74c9bc83088601425e1c3225e6b54de0e0daa9ba67e24f0de5adf67dbbaf393f" gracePeriod=30
Jul 28 01:59:11.427823 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:11.427783    2565 generic.go:356] "Generic (PLEG): container finished" podID="51adf7ef-20cd-402a-bb20-6804f7b8f891" containerID="74c9bc83088601425e1c3225e6b54de0e0daa9ba67e24f0de5adf67dbbaf393f" exitCode=0
Jul 28 01:59:11.428158 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:11.427825    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x" event={"ID":"51adf7ef-20cd-402a-bb20-6804f7b8f891","Type":"ContainerDied","Data":"74c9bc83088601425e1c3225e6b54de0e0daa9ba67e24f0de5adf67dbbaf393f"}
Jul 28 01:59:11.524680 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:11.524627    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:59:11.686174 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:11.686143    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5pfv\" (UniqueName: \"kubernetes.io/projected/51adf7ef-20cd-402a-bb20-6804f7b8f891-kube-api-access-p5pfv\") pod \"51adf7ef-20cd-402a-bb20-6804f7b8f891\" (UID: \"51adf7ef-20cd-402a-bb20-6804f7b8f891\") "
Jul 28 01:59:11.686339 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:11.686276    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51adf7ef-20cd-402a-bb20-6804f7b8f891-openshift-service-ca\") pod \"51adf7ef-20cd-402a-bb20-6804f7b8f891\" (UID: \"51adf7ef-20cd-402a-bb20-6804f7b8f891\") "
Jul 28 01:59:11.686339 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:11.686302    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51adf7ef-20cd-402a-bb20-6804f7b8f891-maas-api-tls\") pod \"51adf7ef-20cd-402a-bb20-6804f7b8f891\" (UID: \"51adf7ef-20cd-402a-bb20-6804f7b8f891\") "
Jul 28 01:59:11.686668 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:11.686647    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51adf7ef-20cd-402a-bb20-6804f7b8f891-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "51adf7ef-20cd-402a-bb20-6804f7b8f891" (UID: "51adf7ef-20cd-402a-bb20-6804f7b8f891"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:59:11.688500 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:11.688457    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51adf7ef-20cd-402a-bb20-6804f7b8f891-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "51adf7ef-20cd-402a-bb20-6804f7b8f891" (UID: "51adf7ef-20cd-402a-bb20-6804f7b8f891"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:59:11.688606 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:11.688567    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51adf7ef-20cd-402a-bb20-6804f7b8f891-kube-api-access-p5pfv" (OuterVolumeSpecName: "kube-api-access-p5pfv") pod "51adf7ef-20cd-402a-bb20-6804f7b8f891" (UID: "51adf7ef-20cd-402a-bb20-6804f7b8f891"). InnerVolumeSpecName "kube-api-access-p5pfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:59:11.787464 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:11.787385    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51adf7ef-20cd-402a-bb20-6804f7b8f891-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:59:11.787464 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:11.787417    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5pfv\" (UniqueName: \"kubernetes.io/projected/51adf7ef-20cd-402a-bb20-6804f7b8f891-kube-api-access-p5pfv\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:59:11.787464 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:11.787431    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51adf7ef-20cd-402a-bb20-6804f7b8f891-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:59:12.433394 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:12.433361    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"
Jul 28 01:59:12.433836 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:12.433394    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x" event={"ID":"51adf7ef-20cd-402a-bb20-6804f7b8f891","Type":"ContainerDied","Data":"8017cd422377a172b0ada751b51bc6fb6e87640fdc20bde98f48c38d34751a7f"}
Jul 28 01:59:12.433836 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:12.433459    2565 scope.go:117] "RemoveContainer" containerID="74c9bc83088601425e1c3225e6b54de0e0daa9ba67e24f0de5adf67dbbaf393f"
Jul 28 01:59:12.457169 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:12.457148    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"]
Jul 28 01:59:12.461248 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:12.461228    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9e1b27-6fb999c47-hxk5x"]
Jul 28 01:59:13.466680 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:13.466652    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51adf7ef-20cd-402a-bb20-6804f7b8f891" path="/var/lib/kubelet/pods/51adf7ef-20cd-402a-bb20-6804f7b8f891/volumes"
Jul 28 01:59:36.444066 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:36.443990    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"]
Jul 28 01:59:36.444599 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:36.444211    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv" podUID="efffbb7c-c61c-4852-8657-78c64c9b0eeb" containerName="maas-api" containerID="cri-o://f8d800421a56541bd96db895eb7eac942b1274aabc90932ec9c6c9fad12a1459" gracePeriod=30
Jul 28 01:59:36.896510 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:36.896451    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:59:36.924096 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:36.924074    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efffbb7c-c61c-4852-8657-78c64c9b0eeb-openshift-service-ca\") pod \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\" (UID: \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\") "
Jul 28 01:59:36.924244 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:36.924115    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sf6dq\" (UniqueName: \"kubernetes.io/projected/efffbb7c-c61c-4852-8657-78c64c9b0eeb-kube-api-access-sf6dq\") pod \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\" (UID: \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\") "
Jul 28 01:59:36.924244 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:36.924139    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efffbb7c-c61c-4852-8657-78c64c9b0eeb-maas-api-tls\") pod \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\" (UID: \"efffbb7c-c61c-4852-8657-78c64c9b0eeb\") "
Jul 28 01:59:36.924462 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:36.924440    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efffbb7c-c61c-4852-8657-78c64c9b0eeb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "efffbb7c-c61c-4852-8657-78c64c9b0eeb" (UID: "efffbb7c-c61c-4852-8657-78c64c9b0eeb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:59:36.926259 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:36.926236    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efffbb7c-c61c-4852-8657-78c64c9b0eeb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "efffbb7c-c61c-4852-8657-78c64c9b0eeb" (UID: "efffbb7c-c61c-4852-8657-78c64c9b0eeb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:59:36.926368 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:36.926345    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efffbb7c-c61c-4852-8657-78c64c9b0eeb-kube-api-access-sf6dq" (OuterVolumeSpecName: "kube-api-access-sf6dq") pod "efffbb7c-c61c-4852-8657-78c64c9b0eeb" (UID: "efffbb7c-c61c-4852-8657-78c64c9b0eeb"). InnerVolumeSpecName "kube-api-access-sf6dq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:59:37.025282 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:37.025192    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efffbb7c-c61c-4852-8657-78c64c9b0eeb-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:59:37.025282 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:37.025231    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sf6dq\" (UniqueName: \"kubernetes.io/projected/efffbb7c-c61c-4852-8657-78c64c9b0eeb-kube-api-access-sf6dq\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:59:37.025282 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:37.025241    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efffbb7c-c61c-4852-8657-78c64c9b0eeb-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:59:37.563032 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:37.562987    2565 generic.go:356] "Generic (PLEG): container finished" podID="efffbb7c-c61c-4852-8657-78c64c9b0eeb" containerID="f8d800421a56541bd96db895eb7eac942b1274aabc90932ec9c6c9fad12a1459" exitCode=0
Jul 28 01:59:37.563413 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:37.563047    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"
Jul 28 01:59:37.563413 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:37.563058    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv" event={"ID":"efffbb7c-c61c-4852-8657-78c64c9b0eeb","Type":"ContainerDied","Data":"f8d800421a56541bd96db895eb7eac942b1274aabc90932ec9c6c9fad12a1459"}
Jul 28 01:59:37.563413 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:37.563135    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv" event={"ID":"efffbb7c-c61c-4852-8657-78c64c9b0eeb","Type":"ContainerDied","Data":"92fa6f91f5a455c848b8ee6a35edcb2deffbdb39bef1bf94adf6f82ef74e45b8"}
Jul 28 01:59:37.563413 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:37.563164    2565 scope.go:117] "RemoveContainer" containerID="f8d800421a56541bd96db895eb7eac942b1274aabc90932ec9c6c9fad12a1459"
Jul 28 01:59:37.572465 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:37.572438    2565 scope.go:117] "RemoveContainer" containerID="f8d800421a56541bd96db895eb7eac942b1274aabc90932ec9c6c9fad12a1459"
Jul 28 01:59:37.572806 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:59:37.572785    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8d800421a56541bd96db895eb7eac942b1274aabc90932ec9c6c9fad12a1459\": container with ID starting with f8d800421a56541bd96db895eb7eac942b1274aabc90932ec9c6c9fad12a1459 not found: ID does not exist" containerID="f8d800421a56541bd96db895eb7eac942b1274aabc90932ec9c6c9fad12a1459"
Jul 28 01:59:37.572864 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:37.572823    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8d800421a56541bd96db895eb7eac942b1274aabc90932ec9c6c9fad12a1459"} err="failed to get container status \"f8d800421a56541bd96db895eb7eac942b1274aabc90932ec9c6c9fad12a1459\": rpc error: code = NotFound desc = could not find container \"f8d800421a56541bd96db895eb7eac942b1274aabc90932ec9c6c9fad12a1459\": container with ID starting with f8d800421a56541bd96db895eb7eac942b1274aabc90932ec9c6c9fad12a1459 not found: ID does not exist"
Jul 28 01:59:37.582209 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:37.582187    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"]
Jul 28 01:59:37.587378 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:37.587357    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8a33ed-5ff4c8c848-jj8bv"]
Jul 28 01:59:39.467536 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:39.467476    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efffbb7c-c61c-4852-8657-78c64c9b0eeb" path="/var/lib/kubelet/pods/efffbb7c-c61c-4852-8657-78c64c9b0eeb/volumes"
Jul 28 01:59:42.071802 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:42.071763    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"]
Jul 28 01:59:57.091426 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.091382    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf" podUID="f8f6dd2b-f49b-4268-8241-4d333e36962c" containerName="main" containerID="cri-o://f6d27f93063fba09346242118af8e829cb23ba398895c7fe596bcec8751c5e85" gracePeriod=45
Jul 28 01:59:57.337929 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.337905    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:59:57.417913 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.417840    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-tmp-dir\") pod \"f8f6dd2b-f49b-4268-8241-4d333e36962c\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") "
Jul 28 01:59:57.417913 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.417870    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-model-cache\") pod \"f8f6dd2b-f49b-4268-8241-4d333e36962c\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") "
Jul 28 01:59:57.417913 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.417903    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-dshm\") pod \"f8f6dd2b-f49b-4268-8241-4d333e36962c\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") "
Jul 28 01:59:57.418188 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.417944    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcppr\" (UniqueName: \"kubernetes.io/projected/f8f6dd2b-f49b-4268-8241-4d333e36962c-kube-api-access-xcppr\") pod \"f8f6dd2b-f49b-4268-8241-4d333e36962c\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") "
Jul 28 01:59:57.418188 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.417975    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8f6dd2b-f49b-4268-8241-4d333e36962c-tls-certs\") pod \"f8f6dd2b-f49b-4268-8241-4d333e36962c\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") "
Jul 28 01:59:57.418188 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.418082    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-home\") pod \"f8f6dd2b-f49b-4268-8241-4d333e36962c\" (UID: \"f8f6dd2b-f49b-4268-8241-4d333e36962c\") "
Jul 28 01:59:57.418188 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.418146    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-model-cache" (OuterVolumeSpecName: "model-cache") pod "f8f6dd2b-f49b-4268-8241-4d333e36962c" (UID: "f8f6dd2b-f49b-4268-8241-4d333e36962c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:59:57.418188 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.418160    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f8f6dd2b-f49b-4268-8241-4d333e36962c" (UID: "f8f6dd2b-f49b-4268-8241-4d333e36962c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:59:57.418437 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.418337    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-home" (OuterVolumeSpecName: "home") pod "f8f6dd2b-f49b-4268-8241-4d333e36962c" (UID: "f8f6dd2b-f49b-4268-8241-4d333e36962c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:59:57.418565 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.418477    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-home\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:59:57.418565 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.418538    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-tmp-dir\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:59:57.418565 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.418554    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-model-cache\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:59:57.420295 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.420271    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-dshm" (OuterVolumeSpecName: "dshm") pod "f8f6dd2b-f49b-4268-8241-4d333e36962c" (UID: "f8f6dd2b-f49b-4268-8241-4d333e36962c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:59:57.420402 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.420299    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8f6dd2b-f49b-4268-8241-4d333e36962c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f8f6dd2b-f49b-4268-8241-4d333e36962c" (UID: "f8f6dd2b-f49b-4268-8241-4d333e36962c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:59:57.420402 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.420337    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8f6dd2b-f49b-4268-8241-4d333e36962c-kube-api-access-xcppr" (OuterVolumeSpecName: "kube-api-access-xcppr") pod "f8f6dd2b-f49b-4268-8241-4d333e36962c" (UID: "f8f6dd2b-f49b-4268-8241-4d333e36962c"). InnerVolumeSpecName "kube-api-access-xcppr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:59:57.519255 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.519230    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8f6dd2b-f49b-4268-8241-4d333e36962c-dshm\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:59:57.519255 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.519253    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcppr\" (UniqueName: \"kubernetes.io/projected/f8f6dd2b-f49b-4268-8241-4d333e36962c-kube-api-access-xcppr\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:59:57.519395 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.519263    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8f6dd2b-f49b-4268-8241-4d333e36962c-tls-certs\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 01:59:57.669462 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.669364    2565 generic.go:356] "Generic (PLEG): container finished" podID="f8f6dd2b-f49b-4268-8241-4d333e36962c" containerID="f6d27f93063fba09346242118af8e829cb23ba398895c7fe596bcec8751c5e85" exitCode=0
Jul 28 01:59:57.669462 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.669407    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf" event={"ID":"f8f6dd2b-f49b-4268-8241-4d333e36962c","Type":"ContainerDied","Data":"f6d27f93063fba09346242118af8e829cb23ba398895c7fe596bcec8751c5e85"}
Jul 28 01:59:57.669462 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.669429    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"
Jul 28 01:59:57.669462 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.669443    2565 scope.go:117] "RemoveContainer" containerID="f6d27f93063fba09346242118af8e829cb23ba398895c7fe596bcec8751c5e85"
Jul 28 01:59:57.669791 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.669433    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf" event={"ID":"f8f6dd2b-f49b-4268-8241-4d333e36962c","Type":"ContainerDied","Data":"1c633de7ca30b810c6f2ec63f509476662d84cda547428fd9053394dfd4dcc04"}
Jul 28 01:59:57.678886 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.678863    2565 scope.go:117] "RemoveContainer" containerID="f6d27f93063fba09346242118af8e829cb23ba398895c7fe596bcec8751c5e85"
Jul 28 01:59:57.679166 ip-10-0-138-173 kubenswrapper[2565]: E0728 01:59:57.679141    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6d27f93063fba09346242118af8e829cb23ba398895c7fe596bcec8751c5e85\": container with ID starting with f6d27f93063fba09346242118af8e829cb23ba398895c7fe596bcec8751c5e85 not found: ID does not exist" containerID="f6d27f93063fba09346242118af8e829cb23ba398895c7fe596bcec8751c5e85"
Jul 28 01:59:57.679218 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.679177    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6d27f93063fba09346242118af8e829cb23ba398895c7fe596bcec8751c5e85"} err="failed to get container status \"f6d27f93063fba09346242118af8e829cb23ba398895c7fe596bcec8751c5e85\": rpc error: code = NotFound desc = could not find container \"f6d27f93063fba09346242118af8e829cb23ba398895c7fe596bcec8751c5e85\": container with ID starting with f6d27f93063fba09346242118af8e829cb23ba398895c7fe596bcec8751c5e85 not found: ID does not exist"
Jul 28 01:59:57.686553 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.686529    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"]
Jul 28 01:59:57.690120 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:57.690102    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-8a33ed/ipp-route-8a33ed-kserve-846679f5bc-spbqf"]
Jul 28 01:59:59.467050 ip-10-0-138-173 kubenswrapper[2565]: I0728 01:59:59.467016    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8f6dd2b-f49b-4268-8241-4d333e36962c" path="/var/lib/kubelet/pods/f8f6dd2b-f49b-4268-8241-4d333e36962c/volumes"
Jul 28 02:00:47.621475 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.621439    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"]
Jul 28 02:00:47.622085 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.621752    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx" podUID="0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23" containerName="maas-api" containerID="cri-o://08197d6e0a0f4e5ad985d6581771610889e9569a450bf8c7ecc6b87cfc294fb5" gracePeriod=30
Jul 28 02:00:47.872850 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.872789    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 02:00:47.914608 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.914285    2565 generic.go:356] "Generic (PLEG): container finished" podID="0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23" containerID="08197d6e0a0f4e5ad985d6581771610889e9569a450bf8c7ecc6b87cfc294fb5" exitCode=0
Jul 28 02:00:47.914608 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.914374    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx" event={"ID":"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23","Type":"ContainerDied","Data":"08197d6e0a0f4e5ad985d6581771610889e9569a450bf8c7ecc6b87cfc294fb5"}
Jul 28 02:00:47.914608 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.914426    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx" event={"ID":"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23","Type":"ContainerDied","Data":"9f9a8e398bfa45f369ab22c1941942a2045a81559872e96814e8e3e03a40a051"}
Jul 28 02:00:47.914608 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.914453    2565 scope.go:117] "RemoveContainer" containerID="08197d6e0a0f4e5ad985d6581771610889e9569a450bf8c7ecc6b87cfc294fb5"
Jul 28 02:00:47.914979 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.914633    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"
Jul 28 02:00:47.923296 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.923274    2565 scope.go:117] "RemoveContainer" containerID="08197d6e0a0f4e5ad985d6581771610889e9569a450bf8c7ecc6b87cfc294fb5"
Jul 28 02:00:47.923583 ip-10-0-138-173 kubenswrapper[2565]: E0728 02:00:47.923559    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08197d6e0a0f4e5ad985d6581771610889e9569a450bf8c7ecc6b87cfc294fb5\": container with ID starting with 08197d6e0a0f4e5ad985d6581771610889e9569a450bf8c7ecc6b87cfc294fb5 not found: ID does not exist" containerID="08197d6e0a0f4e5ad985d6581771610889e9569a450bf8c7ecc6b87cfc294fb5"
Jul 28 02:00:47.923692 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.923596    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08197d6e0a0f4e5ad985d6581771610889e9569a450bf8c7ecc6b87cfc294fb5"} err="failed to get container status \"08197d6e0a0f4e5ad985d6581771610889e9569a450bf8c7ecc6b87cfc294fb5\": rpc error: code = NotFound desc = could not find container \"08197d6e0a0f4e5ad985d6581771610889e9569a450bf8c7ecc6b87cfc294fb5\": container with ID starting with 08197d6e0a0f4e5ad985d6581771610889e9569a450bf8c7ecc6b87cfc294fb5 not found: ID does not exist"
Jul 28 02:00:47.989635 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.989603    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-openshift-service-ca\") pod \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\" (UID: \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\") "
Jul 28 02:00:47.989833 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.989652    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-maas-api-tls\") pod \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\" (UID: \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\") "
Jul 28 02:00:47.989833 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.989691    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhx9b\" (UniqueName: \"kubernetes.io/projected/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-kube-api-access-zhx9b\") pod \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\" (UID: \"0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23\") "
Jul 28 02:00:47.989986 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.989958    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23" (UID: "0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 02:00:47.992065 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.992034    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-kube-api-access-zhx9b" (OuterVolumeSpecName: "kube-api-access-zhx9b") pod "0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23" (UID: "0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23"). InnerVolumeSpecName "kube-api-access-zhx9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 02:00:47.992065 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:47.992043    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23" (UID: "0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 02:00:48.090602 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:48.090562    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 02:00:48.090602 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:48.090595    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 02:00:48.090602 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:48.090604    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhx9b\" (UniqueName: \"kubernetes.io/projected/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23-kube-api-access-zhx9b\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 02:00:48.237463 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:48.237432    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"]
Jul 28 02:00:48.242488 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:48.242444    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c430e4-8487877594-zsbwx"]
Jul 28 02:00:49.467649 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:49.467620    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23" path="/var/lib/kubelet/pods/0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23/volumes"
Jul 28 02:00:59.919941 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:59.919849    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"]
Jul 28 02:00:59.920444 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:59.920090    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd" podUID="9e36e728-24db-4f75-b8ec-49b0ca7b3150" containerName="maas-api" containerID="cri-o://2367e6b360f2053e1ae4865f9e4a2d78b06f571518ccc1e86465013ac874fa4c" gracePeriod=30
Jul 28 02:00:59.994778 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:00:59.994733    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd" podUID="9e36e728-24db-4f75-b8ec-49b0ca7b3150" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.47:8443/health\": dial tcp 10.132.0.47:8443: connect: connection refused"
Jul 28 02:01:00.178777 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.178718    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 02:01:00.301095 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.301067    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bdpr\" (UniqueName: \"kubernetes.io/projected/9e36e728-24db-4f75-b8ec-49b0ca7b3150-kube-api-access-7bdpr\") pod \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\" (UID: \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\") "
Jul 28 02:01:00.301237 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.301129    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e36e728-24db-4f75-b8ec-49b0ca7b3150-openshift-service-ca\") pod \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\" (UID: \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\") "
Jul 28 02:01:00.301237 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.301217    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e36e728-24db-4f75-b8ec-49b0ca7b3150-maas-api-tls\") pod \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\" (UID: \"9e36e728-24db-4f75-b8ec-49b0ca7b3150\") "
Jul 28 02:01:00.301551 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.301528    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e36e728-24db-4f75-b8ec-49b0ca7b3150-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9e36e728-24db-4f75-b8ec-49b0ca7b3150" (UID: "9e36e728-24db-4f75-b8ec-49b0ca7b3150"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 02:01:00.303570 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.303465    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e36e728-24db-4f75-b8ec-49b0ca7b3150-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9e36e728-24db-4f75-b8ec-49b0ca7b3150" (UID: "9e36e728-24db-4f75-b8ec-49b0ca7b3150"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 02:01:00.303570 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.303520    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e36e728-24db-4f75-b8ec-49b0ca7b3150-kube-api-access-7bdpr" (OuterVolumeSpecName: "kube-api-access-7bdpr") pod "9e36e728-24db-4f75-b8ec-49b0ca7b3150" (UID: "9e36e728-24db-4f75-b8ec-49b0ca7b3150"). InnerVolumeSpecName "kube-api-access-7bdpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 02:01:00.402473 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.402433    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e36e728-24db-4f75-b8ec-49b0ca7b3150-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 02:01:00.402473 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.402473    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e36e728-24db-4f75-b8ec-49b0ca7b3150-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 02:01:00.402730 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.402501    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bdpr\" (UniqueName: \"kubernetes.io/projected/9e36e728-24db-4f75-b8ec-49b0ca7b3150-kube-api-access-7bdpr\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 02:01:00.982904 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.982871    2565 generic.go:356] "Generic (PLEG): container finished" podID="9e36e728-24db-4f75-b8ec-49b0ca7b3150" containerID="2367e6b360f2053e1ae4865f9e4a2d78b06f571518ccc1e86465013ac874fa4c" exitCode=0
Jul 28 02:01:00.983314 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.982931    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd" event={"ID":"9e36e728-24db-4f75-b8ec-49b0ca7b3150","Type":"ContainerDied","Data":"2367e6b360f2053e1ae4865f9e4a2d78b06f571518ccc1e86465013ac874fa4c"}
Jul 28 02:01:00.983314 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.982958    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"
Jul 28 02:01:00.983314 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.982973    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd" event={"ID":"9e36e728-24db-4f75-b8ec-49b0ca7b3150","Type":"ContainerDied","Data":"fbf2dd53040c264b100a2507e2c166850e4c5ffedaaf77fb554649ff30ad4cef"}
Jul 28 02:01:00.983314 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.982997    2565 scope.go:117] "RemoveContainer" containerID="2367e6b360f2053e1ae4865f9e4a2d78b06f571518ccc1e86465013ac874fa4c"
Jul 28 02:01:00.992081 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.992063    2565 scope.go:117] "RemoveContainer" containerID="2367e6b360f2053e1ae4865f9e4a2d78b06f571518ccc1e86465013ac874fa4c"
Jul 28 02:01:00.992360 ip-10-0-138-173 kubenswrapper[2565]: E0728 02:01:00.992343    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2367e6b360f2053e1ae4865f9e4a2d78b06f571518ccc1e86465013ac874fa4c\": container with ID starting with 2367e6b360f2053e1ae4865f9e4a2d78b06f571518ccc1e86465013ac874fa4c not found: ID does not exist" containerID="2367e6b360f2053e1ae4865f9e4a2d78b06f571518ccc1e86465013ac874fa4c"
Jul 28 02:01:00.992434 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:00.992369    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2367e6b360f2053e1ae4865f9e4a2d78b06f571518ccc1e86465013ac874fa4c"} err="failed to get container status \"2367e6b360f2053e1ae4865f9e4a2d78b06f571518ccc1e86465013ac874fa4c\": rpc error: code = NotFound desc = could not find container \"2367e6b360f2053e1ae4865f9e4a2d78b06f571518ccc1e86465013ac874fa4c\": container with ID starting with 2367e6b360f2053e1ae4865f9e4a2d78b06f571518ccc1e86465013ac874fa4c not found: ID does not exist"
Jul 28 02:01:01.007852 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:01.007823    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"]
Jul 28 02:01:01.013658 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:01.013636    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0337e8-7ff76f4976-4qxxd"]
Jul 28 02:01:01.466898 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:01.466862    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e36e728-24db-4f75-b8ec-49b0ca7b3150" path="/var/lib/kubelet/pods/9e36e728-24db-4f75-b8ec-49b0ca7b3150/volumes"
Jul 28 02:01:17.623439 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:17.623405    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"]
Jul 28 02:01:17.623864 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:17.623662    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w" podUID="27d96893-46fd-4476-994f-cb08c1f1b02f" containerName="maas-api" containerID="cri-o://683e15c8f55f7cc003b26663333a500fa87dae6bbb91c8b0dfae7e6ee3686cc0" gracePeriod=30
Jul 28 02:01:17.885052 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:17.884993    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 02:01:17.967069 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:17.967035    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d96893-46fd-4476-994f-cb08c1f1b02f-openshift-service-ca\") pod \"27d96893-46fd-4476-994f-cb08c1f1b02f\" (UID: \"27d96893-46fd-4476-994f-cb08c1f1b02f\") "
Jul 28 02:01:17.967069 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:17.967077    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27d96893-46fd-4476-994f-cb08c1f1b02f-maas-api-tls\") pod \"27d96893-46fd-4476-994f-cb08c1f1b02f\" (UID: \"27d96893-46fd-4476-994f-cb08c1f1b02f\") "
Jul 28 02:01:17.967317 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:17.967097    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x87zb\" (UniqueName: \"kubernetes.io/projected/27d96893-46fd-4476-994f-cb08c1f1b02f-kube-api-access-x87zb\") pod \"27d96893-46fd-4476-994f-cb08c1f1b02f\" (UID: \"27d96893-46fd-4476-994f-cb08c1f1b02f\") "
Jul 28 02:01:17.967436 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:17.967414    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27d96893-46fd-4476-994f-cb08c1f1b02f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "27d96893-46fd-4476-994f-cb08c1f1b02f" (UID: "27d96893-46fd-4476-994f-cb08c1f1b02f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 02:01:17.969379 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:17.969350    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27d96893-46fd-4476-994f-cb08c1f1b02f-kube-api-access-x87zb" (OuterVolumeSpecName: "kube-api-access-x87zb") pod "27d96893-46fd-4476-994f-cb08c1f1b02f" (UID: "27d96893-46fd-4476-994f-cb08c1f1b02f"). InnerVolumeSpecName "kube-api-access-x87zb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 02:01:17.969508 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:17.969385    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27d96893-46fd-4476-994f-cb08c1f1b02f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "27d96893-46fd-4476-994f-cb08c1f1b02f" (UID: "27d96893-46fd-4476-994f-cb08c1f1b02f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 02:01:18.067309 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:18.067277    2565 generic.go:356] "Generic (PLEG): container finished" podID="27d96893-46fd-4476-994f-cb08c1f1b02f" containerID="683e15c8f55f7cc003b26663333a500fa87dae6bbb91c8b0dfae7e6ee3686cc0" exitCode=0
Jul 28 02:01:18.067499 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:18.067320    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"
Jul 28 02:01:18.067499 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:18.067373    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w" event={"ID":"27d96893-46fd-4476-994f-cb08c1f1b02f","Type":"ContainerDied","Data":"683e15c8f55f7cc003b26663333a500fa87dae6bbb91c8b0dfae7e6ee3686cc0"}
Jul 28 02:01:18.067499 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:18.067431    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w" event={"ID":"27d96893-46fd-4476-994f-cb08c1f1b02f","Type":"ContainerDied","Data":"2f1bd698a6b07898ae90e758116f715ff16da43bd6d5e92c7effca3a4269ecdf"}
Jul 28 02:01:18.067499 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:18.067459    2565 scope.go:117] "RemoveContainer" containerID="683e15c8f55f7cc003b26663333a500fa87dae6bbb91c8b0dfae7e6ee3686cc0"
Jul 28 02:01:18.067919 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:18.067894    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27d96893-46fd-4476-994f-cb08c1f1b02f-openshift-service-ca\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 02:01:18.067919 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:18.067917    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27d96893-46fd-4476-994f-cb08c1f1b02f-maas-api-tls\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 02:01:18.068067 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:18.067931    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x87zb\" (UniqueName: \"kubernetes.io/projected/27d96893-46fd-4476-994f-cb08c1f1b02f-kube-api-access-x87zb\") on node \"ip-10-0-138-173.ec2.internal\" DevicePath \"\""
Jul 28 02:01:18.077212 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:18.077195    2565 scope.go:117] "RemoveContainer" containerID="683e15c8f55f7cc003b26663333a500fa87dae6bbb91c8b0dfae7e6ee3686cc0"
Jul 28 02:01:18.077471 ip-10-0-138-173 kubenswrapper[2565]: E0728 02:01:18.077454    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"683e15c8f55f7cc003b26663333a500fa87dae6bbb91c8b0dfae7e6ee3686cc0\": container with ID starting with 683e15c8f55f7cc003b26663333a500fa87dae6bbb91c8b0dfae7e6ee3686cc0 not found: ID does not exist" containerID="683e15c8f55f7cc003b26663333a500fa87dae6bbb91c8b0dfae7e6ee3686cc0"
Jul 28 02:01:18.077541 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:18.077510    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"683e15c8f55f7cc003b26663333a500fa87dae6bbb91c8b0dfae7e6ee3686cc0"} err="failed to get container status \"683e15c8f55f7cc003b26663333a500fa87dae6bbb91c8b0dfae7e6ee3686cc0\": rpc error: code = NotFound desc = could not find container \"683e15c8f55f7cc003b26663333a500fa87dae6bbb91c8b0dfae7e6ee3686cc0\": container with ID starting with 683e15c8f55f7cc003b26663333a500fa87dae6bbb91c8b0dfae7e6ee3686cc0 not found: ID does not exist"
Jul 28 02:01:18.091211 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:18.091185    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"]
Jul 28 02:01:18.095013 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:18.094988    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-44b320-66475f898d-74x8w"]
Jul 28 02:01:19.467636 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:19.467593    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27d96893-46fd-4476-994f-cb08c1f1b02f" path="/var/lib/kubelet/pods/27d96893-46fd-4476-994f-cb08c1f1b02f/volumes"
Jul 28 02:01:30.805585 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:30.805558    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk_f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4/copy-manifests/0.log"
Jul 28 02:01:30.817655 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:30.817630    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk_f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4/manager/0.log"
Jul 28 02:01:31.395717 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:31.395675    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-59df996f58-q8ndp_b57dd6cc-85a4-4168-904f-8640c67fef7d/maas-api/0.log"
Jul 28 02:01:33.384558 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:33.384533    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-dhxcx_01f02fd0-c957-42e1-bb0f-4481cf6c30b5/kuadrant-console-plugin/0.log"
Jul 28 02:01:33.609301 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:33.609271    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-j2mk5_864b86be-62aa-4635-804c-d56ef67e55f0/manager/0.log"
Jul 28 02:01:34.522135 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:34.522087    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f6ffdb965-shxvs_ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68/kube-auth-proxy/0.log"
Jul 28 02:01:35.648659 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:35.648630    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fj829k_9857088a-a053-4582-9e7e-faa87bf4f972/main/0.log"
Jul 28 02:01:35.762313 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:35.762281    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-rxpcw_6120e58f-7665-4c20-a550-4a57e1b34ad9/main/0.log"
Jul 28 02:01:35.875093 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:35.875053    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-h4p8q_f3b10cf3-4750-460e-bda7-783e4da98f0b/main/0.log"
Jul 28 02:01:45.255545 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:45.255505    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8k57t_c729200d-5807-4807-a078-990cef3e1bb3/global-pull-secret-syncer/0.log"
Jul 28 02:01:45.432404 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:45.432373    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6p8r2_58d4d35b-d49f-4018-bfdb-f3f4a0ad53a0/konnectivity-agent/0.log"
Jul 28 02:01:45.537790 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:45.537713    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-173.ec2.internal_415ae5ae7d5d3fc074b4ce415469a19e/haproxy/0.log"
Jul 28 02:01:51.608705 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:51.608672    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-dhxcx_01f02fd0-c957-42e1-bb0f-4481cf6c30b5/kuadrant-console-plugin/0.log"
Jul 28 02:01:51.841144 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:51.841121    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-j2mk5_864b86be-62aa-4635-804c-d56ef67e55f0/manager/0.log"
Jul 28 02:01:53.779133 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:53.779101    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/1.log"
Jul 28 02:01:53.848291 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:53.848261    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-gvcqq_2da8692b-45f3-435e-b898-ff3f419c5f4a/cluster-monitoring-operator/0.log"
Jul 28 02:01:53.963546 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:53.963518    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-79755cbbf5-f99cp_4fb40ff7-f19b-4569-83fe-ffe6f49542dc/metrics-server/0.log"
Jul 28 02:01:54.111654 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.111627    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-466qt_dde9b44b-23e1-400e-8ae0-ad8cb7d92759/node-exporter/0.log"
Jul 28 02:01:54.137316 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.137290    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-466qt_dde9b44b-23e1-400e-8ae0-ad8cb7d92759/kube-rbac-proxy/0.log"
Jul 28 02:01:54.174266 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.174237    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-466qt_dde9b44b-23e1-400e-8ae0-ad8cb7d92759/init-textfile/0.log"
Jul 28 02:01:54.268528 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.268495    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-245x2_5e6c8e5c-a396-42b4-9820-101ec6d764b5/kube-rbac-proxy-main/0.log"
Jul 28 02:01:54.301638 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.301610    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-245x2_5e6c8e5c-a396-42b4-9820-101ec6d764b5/kube-rbac-proxy-self/0.log"
Jul 28 02:01:54.337334 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.337307    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-245x2_5e6c8e5c-a396-42b4-9820-101ec6d764b5/openshift-state-metrics/0.log"
Jul 28 02:01:54.396049 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.395964    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88b2580a-9d15-45c2-92be-5ab54f563997/prometheus/0.log"
Jul 28 02:01:54.431235 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.431205    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88b2580a-9d15-45c2-92be-5ab54f563997/config-reloader/0.log"
Jul 28 02:01:54.466204 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.466172    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88b2580a-9d15-45c2-92be-5ab54f563997/thanos-sidecar/0.log"
Jul 28 02:01:54.497913 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.497886    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88b2580a-9d15-45c2-92be-5ab54f563997/kube-rbac-proxy-web/0.log"
Jul 28 02:01:54.534288 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.534263    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88b2580a-9d15-45c2-92be-5ab54f563997/kube-rbac-proxy/0.log"
Jul 28 02:01:54.570032 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.570007    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88b2580a-9d15-45c2-92be-5ab54f563997/kube-rbac-proxy-thanos/0.log"
Jul 28 02:01:54.611450 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.611424    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88b2580a-9d15-45c2-92be-5ab54f563997/init-config-reloader/0.log"
Jul 28 02:01:54.729494 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.729409    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cccb99ffb-kmjvv_95135ee8-17bd-461f-9939-5e7cfb6bfa10/telemeter-client/0.log"
Jul 28 02:01:54.766280 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.766248    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cccb99ffb-kmjvv_95135ee8-17bd-461f-9939-5e7cfb6bfa10/reload/0.log"
Jul 28 02:01:54.802078 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:54.802049    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cccb99ffb-kmjvv_95135ee8-17bd-461f-9939-5e7cfb6bfa10/kube-rbac-proxy/0.log"
Jul 28 02:01:56.550243 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550207    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"]
Jul 28 02:01:56.550664 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550621    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23" containerName="maas-api"
Jul 28 02:01:56.550664 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550640    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23" containerName="maas-api"
Jul 28 02:01:56.550664 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550661    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27d96893-46fd-4476-994f-cb08c1f1b02f" containerName="maas-api"
Jul 28 02:01:56.550789 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550670    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="27d96893-46fd-4476-994f-cb08c1f1b02f" containerName="maas-api"
Jul 28 02:01:56.550789 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550681    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8f6dd2b-f49b-4268-8241-4d333e36962c" containerName="main"
Jul 28 02:01:56.550789 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550690    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8f6dd2b-f49b-4268-8241-4d333e36962c" containerName="main"
Jul 28 02:01:56.550789 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550712    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51adf7ef-20cd-402a-bb20-6804f7b8f891" containerName="maas-api"
Jul 28 02:01:56.550789 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550721    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="51adf7ef-20cd-402a-bb20-6804f7b8f891" containerName="maas-api"
Jul 28 02:01:56.550789 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550734    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e36e728-24db-4f75-b8ec-49b0ca7b3150" containerName="maas-api"
Jul 28 02:01:56.550789 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550741    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e36e728-24db-4f75-b8ec-49b0ca7b3150" containerName="maas-api"
Jul 28 02:01:56.550789 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550749    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efffbb7c-c61c-4852-8657-78c64c9b0eeb" containerName="maas-api"
Jul 28 02:01:56.550789 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550757    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="efffbb7c-c61c-4852-8657-78c64c9b0eeb" containerName="maas-api"
Jul 28 02:01:56.551142 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550858    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="efffbb7c-c61c-4852-8657-78c64c9b0eeb" containerName="maas-api"
Jul 28 02:01:56.551142 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550869    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e36e728-24db-4f75-b8ec-49b0ca7b3150" containerName="maas-api"
Jul 28 02:01:56.551142 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550879    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0901efc1-9ce6-46f8-8fb4-f5e3f73e3f23" containerName="maas-api"
Jul 28 02:01:56.551142 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550888    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8f6dd2b-f49b-4268-8241-4d333e36962c" containerName="main"
Jul 28 02:01:56.551142 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550898    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="27d96893-46fd-4476-994f-cb08c1f1b02f" containerName="maas-api"
Jul 28 02:01:56.551142 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.550908    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="51adf7ef-20cd-402a-bb20-6804f7b8f891" containerName="maas-api"
Jul 28 02:01:56.554030 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.554014    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.556992 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.556968    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pzk5p\"/\"kube-root-ca.crt\""
Jul 28 02:01:56.557135 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.556996    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pzk5p\"/\"openshift-service-ca.crt\""
Jul 28 02:01:56.557135 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.557028    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pzk5p\"/\"default-dockercfg-wjwmd\""
Jul 28 02:01:56.563402 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.563374    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"]
Jul 28 02:01:56.609652 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.609616    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fc4e170d-9488-4fa8-bf58-f80a206afe2a-podres\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.609831 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.609674    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fc4e170d-9488-4fa8-bf58-f80a206afe2a-proc\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.609831 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.609725    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc4e170d-9488-4fa8-bf58-f80a206afe2a-lib-modules\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.609831 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.609772    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc4e170d-9488-4fa8-bf58-f80a206afe2a-sys\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.609956 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.609843    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4wgc\" (UniqueName: \"kubernetes.io/projected/fc4e170d-9488-4fa8-bf58-f80a206afe2a-kube-api-access-d4wgc\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.710941 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.710889    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4wgc\" (UniqueName: \"kubernetes.io/projected/fc4e170d-9488-4fa8-bf58-f80a206afe2a-kube-api-access-d4wgc\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.711134 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.710960    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fc4e170d-9488-4fa8-bf58-f80a206afe2a-podres\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.711134 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.710999    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fc4e170d-9488-4fa8-bf58-f80a206afe2a-proc\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.711134 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.711014    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc4e170d-9488-4fa8-bf58-f80a206afe2a-lib-modules\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.711134 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.711085    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc4e170d-9488-4fa8-bf58-f80a206afe2a-sys\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.711134 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.711100    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fc4e170d-9488-4fa8-bf58-f80a206afe2a-proc\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.711326 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.711146    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fc4e170d-9488-4fa8-bf58-f80a206afe2a-podres\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.711326 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.711160    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc4e170d-9488-4fa8-bf58-f80a206afe2a-sys\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.711326 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.711150    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc4e170d-9488-4fa8-bf58-f80a206afe2a-lib-modules\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.718713 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.718690    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/1.log"
Jul 28 02:01:56.720114 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.720090    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4wgc\" (UniqueName: \"kubernetes.io/projected/fc4e170d-9488-4fa8-bf58-f80a206afe2a-kube-api-access-d4wgc\") pod \"perf-node-gather-daemonset-xnplf\" (UID: \"fc4e170d-9488-4fa8-bf58-f80a206afe2a\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:56.723743 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.723723    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9mrwl_e2652d05-c03c-4016-a91d-0e7b279aaa74/console-operator/2.log"
Jul 28 02:01:56.865791 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:56.865762    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:57.004559 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:57.004532    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"]
Jul 28 02:01:57.006984 ip-10-0-138-173 kubenswrapper[2565]: W0728 02:01:57.006953    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfc4e170d_9488_4fa8_bf58_f80a206afe2a.slice/crio-a16cec780f0d1b3e50a915a81404eba30ea6763793bf3f563c031cd2f7b451b8 WatchSource:0}: Error finding container a16cec780f0d1b3e50a915a81404eba30ea6763793bf3f563c031cd2f7b451b8: Status 404 returned error can't find the container with id a16cec780f0d1b3e50a915a81404eba30ea6763793bf3f563c031cd2f7b451b8
Jul 28 02:01:57.216806 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:57.216699    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7b9d549b4-lqlbb_2228e675-8282-41d9-8905-c5505b17318a/console/0.log"
Jul 28 02:01:57.250357 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:57.250310    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-6rl9b_5a341e2a-bbb3-4755-9856-c7fcefe35a59/download-server/0.log"
Jul 28 02:01:57.264433 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:57.264401    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf" event={"ID":"fc4e170d-9488-4fa8-bf58-f80a206afe2a","Type":"ContainerStarted","Data":"838d3f68bc8bb52741ab64389803b67cd5d4bd480925390df4e785e28462fd3c"}
Jul 28 02:01:57.264756 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:57.264460    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf" event={"ID":"fc4e170d-9488-4fa8-bf58-f80a206afe2a","Type":"ContainerStarted","Data":"a16cec780f0d1b3e50a915a81404eba30ea6763793bf3f563c031cd2f7b451b8"}
Jul 28 02:01:57.264756 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:57.264645    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:01:57.286355 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:57.286296    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf" podStartSLOduration=1.286280686 podStartE2EDuration="1.286280686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 02:01:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 02:01:57.283968151 +0000 UTC m=+3468.328862155" watchObservedRunningTime="2026-07-28 02:01:57.286280686 +0000 UTC m=+3468.331174654"
Jul 28 02:01:58.852712 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:58.852672    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4pjfd_1d7ebd81-b836-4d70-926d-52fbc1e6eb80/dns/0.log"
Jul 28 02:01:58.880791 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:58.880764    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4pjfd_1d7ebd81-b836-4d70-926d-52fbc1e6eb80/kube-rbac-proxy/0.log"
Jul 28 02:01:59.064930 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:59.064901    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w4hf9_e09b1d16-8eac-4782-9e73-8a891ab18686/dns-node-resolver/0.log"
Jul 28 02:01:59.630995 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:01:59.630966    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dg6qv_6f874e33-77cd-4288-abd3-77433942b384/node-ca/0.log"
Jul 28 02:02:00.828126 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:00.828096    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f6ffdb965-shxvs_ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68/kube-auth-proxy/0.log"
Jul 28 02:02:01.804837 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:01.804809    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nbqk2_2be7258c-f1ce-42df-8769-90258eda6a2b/serve-healthcheck-canary/0.log"
Jul 28 02:02:02.476048 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:02.475970    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qq72q_dfbb26c6-b5a3-4a63-abae-d63809b5d362/kube-rbac-proxy/0.log"
Jul 28 02:02:02.507114 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:02.507087    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qq72q_dfbb26c6-b5a3-4a63-abae-d63809b5d362/exporter/0.log"
Jul 28 02:02:02.531051 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:02.531023    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qq72q_dfbb26c6-b5a3-4a63-abae-d63809b5d362/extractor/0.log"
Jul 28 02:02:03.279384 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:03.279350    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-xnplf"
Jul 28 02:02:05.373894 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:05.373861    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk_f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4/manager/0.log"
Jul 28 02:02:05.395445 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:05.395422    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5bf98dfb-8qnqk_f2fae8da-c7f3-44b9-9cee-a3cfaa975ef4/copy-manifests/0.log"
Jul 28 02:02:06.019868 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:06.019789    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f6ffdb965-shxvs_ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68/kube-auth-proxy/0.log"
Jul 28 02:02:06.948856 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:06.948821    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6d99f9899c-ndlk2_d144fcce-f5ef-4688-b7d5-70a518a27b7a/manager/0.log"
Jul 28 02:02:07.782670 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:07.782626    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f6ffdb965-shxvs_ea9c1c2d-1ed4-4fe5-a2c0-eaff0403ae68/kube-auth-proxy/0.log"
Jul 28 02:02:13.325853 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:13.325824    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-74qj2_9d8a1856-91c6-49a5-88c7-6e18c5a37a23/kube-multus/0.log"
Jul 28 02:02:13.674802 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:13.674723    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xx4zt_e8d46b27-f0c3-4db4-92fc-1c8d457060fd/kube-multus-additional-cni-plugins/0.log"
Jul 28 02:02:13.696285 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:13.696262    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xx4zt_e8d46b27-f0c3-4db4-92fc-1c8d457060fd/egress-router-binary-copy/0.log"
Jul 28 02:02:13.720695 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:13.720668    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xx4zt_e8d46b27-f0c3-4db4-92fc-1c8d457060fd/cni-plugins/0.log"
Jul 28 02:02:13.742130 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:13.742099    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xx4zt_e8d46b27-f0c3-4db4-92fc-1c8d457060fd/bond-cni-plugin/0.log"
Jul 28 02:02:13.765453 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:13.765429    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xx4zt_e8d46b27-f0c3-4db4-92fc-1c8d457060fd/routeoverride-cni/0.log"
Jul 28 02:02:13.788254 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:13.788225    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xx4zt_e8d46b27-f0c3-4db4-92fc-1c8d457060fd/whereabouts-cni-bincopy/0.log"
Jul 28 02:02:13.809249 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:13.809228    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xx4zt_e8d46b27-f0c3-4db4-92fc-1c8d457060fd/whereabouts-cni/0.log"
Jul 28 02:02:13.926804 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:13.926727    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-g8zj6_f53d5a5f-7a61-468f-8fd6-09912822227d/network-metrics-daemon/0.log"
Jul 28 02:02:13.948093 ip-10-0-138-173 kubenswrapper[2565]: I0728 02:02:13.948067    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-g8zj6_f53d5a5f-7a61-468f-8fd6-09912822227d/kube-rbac-proxy/0.log"