Jul 06 17:38:26.289998 ip-10-0-133-236 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 06 17:38:26.290010 ip-10-0-133-236 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 06 17:38:26.290018 ip-10-0-133-236 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 06 17:38:26.290294 ip-10-0-133-236 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 06 17:38:36.452920 ip-10-0-133-236 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Jul 06 17:38:36.452938 ip-10-0-133-236 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 575aea1fd7dc438ba7e92c9f0bd7d985 --
Jul 06 17:40:24.282139 ip-10-0-133-236 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 17:40:24.764340 ip-10-0-133-236 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 17:40:24.764340 ip-10-0-133-236 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 17:40:24.764340 ip-10-0-133-236 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 17:40:24.764340 ip-10-0-133-236 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 17:40:24.764340 ip-10-0-133-236 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 17:40:24.764340 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.671706    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 17:40:24.764340 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677443    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 17:40:24.764340 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677456    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 17:40:24.764340 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677461    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 17:40:24.764340 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677466    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 17:40:24.764340 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677470    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 17:40:24.764340 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677473    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 17:40:24.764340 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677476    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 17:40:24.764340 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677479    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 17:40:24.744943 ip-10-0-133-236 systemd[1]: Started Kubernetes Kubelet.
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677482    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677485    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677487    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677490    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677492    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677495    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677497    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677500    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677503    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677505    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677509    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677511    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677514    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677517    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677520    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677523    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677525    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677528    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 17:40:24.767403 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677531    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677534    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677536    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677539    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677541    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677544    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677546    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677549    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677551    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677554    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677556    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677559    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677563    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677567    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677570    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677573    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677576    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677580    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677584    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 17:40:24.780368 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677588    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677593    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677596    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677600    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677603    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677605    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677608    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677611    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677614    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677617    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677619    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677624    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677626    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677630    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677632    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677635    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677638    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677640    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677643    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677646    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 17:40:24.781191 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677648    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677651    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677653    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677656    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677658    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677661    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677663    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677666    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677669    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677677    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677680    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677683    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677686    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677689    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677692    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677694    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677698    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677701    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677703    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677706    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 17:40:24.781828 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677709    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677712    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677714    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677717    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677720    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677723    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677725    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677728    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677730    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677733    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677736    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677738    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677741    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.677743    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678426    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678432    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678436    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678439    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678442    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678445    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 17:40:24.782645 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678448    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678451    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678454    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678458    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678461    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678463    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678466    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678469    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678472    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678474    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678477    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678480    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678482    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678485    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678487    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678490    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678493    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678495    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678497    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678500    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 17:40:24.783255 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678503    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678505    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678508    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678510    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678513    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678515    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678518    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678520    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678523    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678525    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678528    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678530    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678536    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678538    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678541    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678544    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678547    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678550    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678554    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 17:40:24.783936 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678558    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678561    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678564    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678567    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678569    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678572    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678574    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678578    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678580    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678583    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678585    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678588    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678591    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678593    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678596    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678598    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678601    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678603    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678606    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678609    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 17:40:24.784497 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678611    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678614    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678616    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678619    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678621    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678625    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678628    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678630    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678633    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678635    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678638    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678641    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678645    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678648    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678650    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678654    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678657    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678660    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678662    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 17:40:24.785164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678665    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678668    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678671    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678673    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678676    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678678    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678683    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678685    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678688    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678690    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678692    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678695    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678698    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678700    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.678703    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679067    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679073    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679076    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679081    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679084    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 17:40:24.785662 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679087    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679090    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679094    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679142    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679284    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679290    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679296    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679300    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679305    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679310    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679315    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679320    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679325    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679330    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679335    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679339    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679344    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679355    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679361    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679365    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 17:40:24.786304 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679370    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679374    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679379    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679383    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679388    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679392    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679396    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679405    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679416    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679421    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679427    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679431    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679435    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679440    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679444    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679448    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679458    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679463    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679467    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 17:40:24.786955 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679471    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679475    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679484    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679488    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679492    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679496    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679500    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679505    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679509    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679513    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679517    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679523    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679527    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679531    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679535    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679544    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679548    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679552    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679557    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679561    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 17:40:24.787537 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679565    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679569    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679573    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679579    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679583    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679587    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679591    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679598    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679608    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679613    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679619    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679626    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679631    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679635    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679639    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679644    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679648    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679653    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679657    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679667    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 17:40:24.788076 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679671    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679676    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679681    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679687    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679691    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679695    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679700    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679704    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679708    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679712    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679716    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679726    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679730    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679734    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.679738    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679857    2581 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679909    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679919    2581 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679925    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679932    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679937    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 17:40:24.788615 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679943    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679949    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679969    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679974    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679980    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679985    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679990    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679995    2581 flags.go:64] FLAG: --cgroup-root=""
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.679999    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680005    2581 flags.go:64] FLAG: --client-ca-file=""
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680009    2581 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680019    2581 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680026    2581 flags.go:64] FLAG: --cluster-domain=""
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680031    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680040    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680043    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680047    2581 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680051    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680054    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680057    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680060    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680064    2581 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680067    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680136    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 17:40:24.789219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680274    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680280    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680285    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680288    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680293    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680296    2581 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680299    2581 flags.go:64] FLAG: --enable-server="true"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680302    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680307    2581 flags.go:64] FLAG: --event-burst="100"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680311    2581 flags.go:64] FLAG: --event-qps="50"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680314    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680317    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680320    2581 flags.go:64] FLAG: --eviction-hard=""
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680323    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680326    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680330    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680333    2581 flags.go:64] FLAG: --eviction-soft=""
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680336    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680339    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680342    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680345    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680347    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680350    2581 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680353    2581 flags.go:64] FLAG: --feature-gates=""
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680360    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 17:40:24.790213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680363    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680366    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680370    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680374    2581 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680377    2581 flags.go:64] FLAG: --help="false"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680379    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-133-236.ec2.internal"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680383    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680386    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680389    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680392    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680396    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680402    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680404    2581 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680407    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680410    2581 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680413    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680417    2581 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680419    2581 flags.go:64] FLAG: --kube-reserved=""
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680422    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680425    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680428    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680430    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680433    2581 flags.go:64] FLAG: --lock-file=""
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680436    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 17:40:24.790919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680439    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680442    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680447    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680450    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680453    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680456    2581 flags.go:64] FLAG: --logging-format="text"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680459    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680463    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680466    2581 flags.go:64] FLAG: --manifest-url=""
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680469    2581 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680474    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680476    2581 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680481    2581 flags.go:64] FLAG: --max-pods="110"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680484    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680487    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680489    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680492    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680495    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680498    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680502    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680510    2581 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680513    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680516    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 17:40:24.791743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680520    2581 flags.go:64] FLAG: --pod-cidr=""
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680522    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680527    2581 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680530    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680533    2581 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680536    2581 flags.go:64] FLAG: --port="10250"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680539    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680542    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a71b638cfb52e9ac"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680545    2581 flags.go:64] FLAG: --qos-reserved=""
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680548    2581 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680551    2581 flags.go:64] FLAG: --register-node="true"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680554    2581 flags.go:64] FLAG: --register-with-taints=""
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680557    2581 flags.go:64] FLAG: --registry-burst="10"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680560    2581 flags.go:64] FLAG: --registry-qps="5"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680563    2581 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680565    2581 flags.go:64] FLAG: --reserved-memory=""
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680570    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680573    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680576    2581 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680579    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680582    2581 flags.go:64] FLAG: --runonce="false"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680585    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680588    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680591    2581 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680593    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 17:40:24.792361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680596    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680599    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680602    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680605    2581 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680610    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680613    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680616    2581 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680619    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680622    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680625    2581 flags.go:64] FLAG: --system-cgroups=""
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680628    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680634    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680637    2581 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680639    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680644    2581 flags.go:64] FLAG: --tls-min-version=""
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680646    2581 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680649    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680652    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680655    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680658    2581 flags.go:64] FLAG: --v="2"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680661    2581 flags.go:64] FLAG: --version="false"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680665    2581 flags.go:64] FLAG: --vmodule=""
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680669    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.680673    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680783    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 17:40:24.793160 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680786    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680790    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680793    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680796    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680799    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680802    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680805    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680808    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680811    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680814    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680816    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680820    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680822    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680825    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680827    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680830    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680832    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680837    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680841    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 17:40:24.794884 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680843    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680846    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680849    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680852    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680855    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680858    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680861    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680864    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680867    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680870    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680873    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680877    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680879    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680882    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680885    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680888    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680905    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680908    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680911    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680914    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 17:40:24.966941 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680916    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680919    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680922    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680925    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680928    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680931    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680933    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680936    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680938    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680941    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680943    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680946    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680949    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680952    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680954    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680957    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680960    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680963    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680965    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 17:40:24.986526 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680968    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680971    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680973    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680976    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680980    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680983    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680985    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680988    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680991    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680993    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680996    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.680998    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681001    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681004    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681006    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681009    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681011    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681017    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681020    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681023    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 17:40:24.987166 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681025    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681028    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681031    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681033    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681036    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681039    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681041    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681044    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681047    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681049    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681052    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681054    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681059    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681062    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681065    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681068    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681072    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681074    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681077    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 17:40:24.987761 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.681080    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.681559    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.687315    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.687328    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.687344    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.687349    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687401    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687405    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687409    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687412    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687415    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687417    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687420    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687423    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687426    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 17:40:24.988341 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687429    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687431    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687434    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687437    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687439    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687442    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687446    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687449    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687452    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687454    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687457    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687459    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687462    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687465    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687469    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687473    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687476    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687480    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687484    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687488    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 17:40:24.988778 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687492    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687497    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687499    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687502    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687506    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687508    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687512    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687515    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687517    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687520    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687522    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687525    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687529    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687532    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687536    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687538    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687541    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687544    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687548    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687551    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 17:40:24.989384 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687554    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687556    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687559    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687561    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687564    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687567    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687569    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687571    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687574    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687577    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687579    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687582    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687584    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687587    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687589    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687592    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687596    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687600    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687602    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687605    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 17:40:24.989946 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687608    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687610    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687613    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687616    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687619    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687623    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687626    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687628    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687631    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687634    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687638    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687640    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687643    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687645    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687648    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687651    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687653    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687655    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687660    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 17:40:24.990529 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687663    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687666    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687668    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687671    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687674    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687676    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687679    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687681    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687684    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687687    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687689    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.687694    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687798    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687801    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687804    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 17:40:24.991063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687807    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687810    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687812    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687816    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687819    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687822    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687824    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687827    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687831    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687835    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687838    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687841    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687844    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687846    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687849    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687851    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687854    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687856    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687860    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 17:40:24.991522 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687863    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687866    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687869    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687872    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687874    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687876    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687879    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687882    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687884    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687887    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687890    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687906    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687909    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687911    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687914    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687916    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687920    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687923    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687925    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687928    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687930    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 17:40:24.992583 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687933    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687937    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687939    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687942    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687945    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687947    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687950    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687952    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687955    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687957    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687960    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687963    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687966    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687969    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687971    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687973    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687976    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687979    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687981    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687984    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 17:40:24.993210 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687986    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687989    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687992    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687994    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687997    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.687999    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688002    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688005    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688008    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688010    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688013    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688015    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688018    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688022    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688024    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688027    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688029    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688032    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688034    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688037    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 17:40:24.993756 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688039    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688042    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688044    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688047    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688049    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688052    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688054    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688057    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688059    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688062    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688064    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688067    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688069    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688072    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688074    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 17:40:24.994378 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:24.688077    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 17:40:24.994783 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.688082    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 17:40:24.994783 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.688167    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 17:40:24.994783 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.690008    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 17:40:24.994783 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.691437    2581 server.go:1019] "Starting client certificate rotation"
Jul 06 17:40:24.994783 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.691531    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 17:40:24.994783 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.691557    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 17:40:24.994783 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.693754    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 17:40:24.994783 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.697669    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 17:40:24.994783 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.709549    2581 log.go:25] "Validated CRI v1 runtime API"
Jul 06 17:40:24.994783 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.712970    2581 log.go:25] "Validated CRI v1 image API"
Jul 06 17:40:24.994783 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.713830    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 17:40:24.994783 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.717991    2581 fs.go:135] Filesystem UUIDs: map[6010a6d9-2fe3-465c-9a80-81bad061535f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 88164bc3-e441-4f96-90bd-40b5e756d47d:/dev/nvme0n1p3]
Jul 06 17:40:24.995194 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.718008    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 17:40:24.995194 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.719228    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.724269    2581 manager.go:216] Machine: {Timestamp:2026-07-06 17:40:24.72270208 +0000 UTC m=+0.316156603 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100693 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2004dda23b17c086598155357d6405 SystemUUID:ec2004dd-a23b-17c0-8659-8155357d6405 BootID:575aea1f-d7dc-438b-a7e9-2c9f0bd7d985 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:bb:97:9f:6e:87 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:bb:97:9f:6e:87 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:e9:8f:61:eb:9c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.724386    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.724467    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.727796    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.727818    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-236.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.728560    2581 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.728571    2581 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.728577    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.729304    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.730557    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.734104    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.734204    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.734229    2581 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.734238    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.734250    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.734262    2581 kubelet.go:398] "Adding apiserver pod source"
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.734270    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 17:40:25.152063 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.734574    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.734590    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.737349    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.739712    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.739739    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.740635    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.740655    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.740663    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.740669    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.740676    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.740683    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.740690    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.740700    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.740712    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.740723    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.740738    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.740746    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.741256    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.741268    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.744236    2581 server.go:1268] "Started kubelet"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.744768    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.744838    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.746718    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.747951    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-236.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:24.748060    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:24.748037    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.748854    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.751488    2581 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:24.751784    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.753348    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.753366    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:24.753426    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:24.753019    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-236.ec2.internal.18bfc4214ba702a3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-236.ec2.internal,UID:ip-10-0-133-236.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-236.ec2.internal,},FirstTimestamp:2026-07-06 17:40:24.744207011 +0000 UTC m=+0.337661542,LastTimestamp:2026-07-06 17:40:24.744207011 +0000 UTC m=+0.337661542,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-236.ec2.internal,}"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.753960    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.753975    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.754391    2581 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.754402    2581 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.754438    2581 factory.go:55] Registering systemd factory
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.754455    2581 factory.go:223] Registration of the systemd container factory successfully
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.754145    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.755007    2581 factory.go:153] Registering CRI-O factory
Jul 06 17:40:25.153220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.755023    2581 factory.go:223] Registration of the crio container factory successfully
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.755071    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.755095    2581 factory.go:103] Registering Raw factory
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.755107    2581 manager.go:1196] Started watching for new ooms in manager
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.755422    2581 manager.go:349] Starting recovery of all containers
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:24.761420    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:24.761481    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.786297    2581 manager.go:354] Recovery completed
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.790778    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.794710    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b8wn5"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.795267    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.795297    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.795311    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasSufficientPID"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.795806    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.795818    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.795808    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.795835    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.796943    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.796971    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.796981    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasSufficientPID"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.796948    2581 policy_none.go:49] "None policy: Start"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.797041    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.797055    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:24.797511    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-236.ec2.internal.18bfc4214eb252bb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-236.ec2.internal,UID:ip-10-0-133-236.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-236.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-236.ec2.internal,},FirstTimestamp:2026-07-06 17:40:24.795280059 +0000 UTC m=+0.388734580,LastTimestamp:2026-07-06 17:40:24.795280059 +0000 UTC m=+0.388734580,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-236.ec2.internal,}"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.797930    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b8wn5"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.797969    2581 policy_none.go:47] "Start"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.846413    2581 manager.go:337] "Starting Device Plugin manager"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:24.846443    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.846453    2581 server.go:87] "Starting device plugin registration server"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.846626    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.846637    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.846773    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.846856    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.846864    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:24.847306    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:24.847341    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.904560    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.905690    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.905726    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 17:40:25.154426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.905747    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:24.905781    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.908232    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.947653    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.948352    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.948373    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.948382    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasSufficientPID"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.948405    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:24.957008    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:24.957023    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-236.ec2.internal\": node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:24.975474    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.006797    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-236.ec2.internal"]
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.006844    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.007968    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.007989    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.008000    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasSufficientPID"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.009394    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.009508    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.009529    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.010106    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.010136    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.010150    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasSufficientPID"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.010371    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.010391    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.010404    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasSufficientPID"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.011609    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.011633    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.012227    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.012253    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.012266    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeHasSufficientPID"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:25.016994    2581 helpers.go:245] readString: Failed to read "/sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28add01306b01a8c01388eec6d6c37cf.slice/cpu.max": open /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28add01306b01a8c01388eec6d6c37cf.slice/cpu.max: no such device
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:25.050462    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-236.ec2.internal\" not found" node="ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:25.058232    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-236.ec2.internal\" not found" node="ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.155876 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:25.075628    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:25.362700 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.158394    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/28add01306b01a8c01388eec6d6c37cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal\" (UID: \"28add01306b01a8c01388eec6d6c37cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.362700 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.158415    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28add01306b01a8c01388eec6d6c37cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal\" (UID: \"28add01306b01a8c01388eec6d6c37cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.362700 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.158430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f393a703fb7ce0f536ef1eee45ebaae3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-236.ec2.internal\" (UID: \"f393a703fb7ce0f536ef1eee45ebaae3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.362700 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:25.176480    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:25.362700 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.258881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/28add01306b01a8c01388eec6d6c37cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal\" (UID: \"28add01306b01a8c01388eec6d6c37cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.362700 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.258921    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28add01306b01a8c01388eec6d6c37cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal\" (UID: \"28add01306b01a8c01388eec6d6c37cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.362700 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.258941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f393a703fb7ce0f536ef1eee45ebaae3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-236.ec2.internal\" (UID: \"f393a703fb7ce0f536ef1eee45ebaae3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.362700 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.258991    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28add01306b01a8c01388eec6d6c37cf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal\" (UID: \"28add01306b01a8c01388eec6d6c37cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.362700 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.258991    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/28add01306b01a8c01388eec6d6c37cf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal\" (UID: \"28add01306b01a8c01388eec6d6c37cf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.362700 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.259021    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f393a703fb7ce0f536ef1eee45ebaae3-config\") pod \"kube-apiserver-proxy-ip-10-0-133-236.ec2.internal\" (UID: \"f393a703fb7ce0f536ef1eee45ebaae3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.362700 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:25.277346    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:25.363121 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.362725    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.363121 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.362744    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:25.547166 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:25.378491    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:25.547166 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:25.479151    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:25.749891 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:25.579947    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:25.749891 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:25.680616    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:25.749891 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.690463    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 17:40:25.749891 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.690565    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 17:40:25.749891 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.729451    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 17:40:25.922475 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:25.780761    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:25.922475 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.800028    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 17:35:25 +0000 UTC" deadline="2028-01-13 04:32:17.339110088 +0000 UTC"
Jul 06 17:40:25.922475 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.800050    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13330h51m51.539062853s"
Jul 06 17:40:25.922475 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.802676    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 17:40:25.922475 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:25.881846    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:26.085242 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:25.927826    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28add01306b01a8c01388eec6d6c37cf.slice/crio-0d3a72ff2f322b17f98f85f2efe7a538cd507960be7bcca9be90562f8c3a2218 WatchSource:0}: Error finding container 0d3a72ff2f322b17f98f85f2efe7a538cd507960be7bcca9be90562f8c3a2218: Status 404 returned error can't find the container with id 0d3a72ff2f322b17f98f85f2efe7a538cd507960be7bcca9be90562f8c3a2218
Jul 06 17:40:26.085242 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:25.928230    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf393a703fb7ce0f536ef1eee45ebaae3.slice/crio-00fcdc0d840887feaca328f1987fcc853e099d2a0e74ddf99786b08e87050b13 WatchSource:0}: Error finding container 00fcdc0d840887feaca328f1987fcc853e099d2a0e74ddf99786b08e87050b13: Status 404 returned error can't find the container with id 00fcdc0d840887feaca328f1987fcc853e099d2a0e74ddf99786b08e87050b13
Jul 06 17:40:26.085242 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:25.932883    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 17:40:26.085242 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:25.982667    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:26.085242 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:26.083483    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-236.ec2.internal\" not found"
Jul 06 17:40:26.267444 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.180274    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 17:40:26.267444 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.254546    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:26.269959 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.268616    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 17:40:26.269959 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.269768    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-236.ec2.internal"
Jul 06 17:40:26.451509 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.278066    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.736062    2581 apiserver.go:52] "Watching apiserver"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.746889    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.747302    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-v58n8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal","openshift-multus/multus-4k5bv","openshift-multus/multus-additional-cni-plugins-mfpln","openshift-multus/network-metrics-daemon-fsm9w","openshift-network-diagnostics/network-check-target-h7n2n","kube-system/konnectivity-agent-dzlq8","kube-system/kube-apiserver-proxy-ip-10-0-133-236.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2","openshift-image-registry/node-ca-5gcxt","openshift-network-operator/iptables-alerter-x5lfm","openshift-ovn-kubernetes/ovnkube-node-xqg68","kube-system/global-pull-secret-syncer-vpff9","openshift-cluster-node-tuning-operator/tuned-fmknh"]
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.749138    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dzlq8"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.750399    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:26.750466    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.751437    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.751642    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kqd5g\""
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.751782    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.752206    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.752859    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.753948    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.754261    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.754512    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.754935    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-49dxz\""
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.757298    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.760350    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:26.760433    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.760668    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zf5v8\""
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.760891    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.761380    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.762072    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.763800    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:26.763905    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.764552    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.765730    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v58n8"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.767414    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.767421    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.767414    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 17:40:26.770264 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.767533    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g6k2d\""
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.767556    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.768175    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.768991    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sht4m\""
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.769863    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.770386    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.770768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.770799    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-os-release\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.770826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b00296d-a49b-4174-b08f-191b31d4c7a5-agent-certs\") pod \"konnectivity-agent-dzlq8\" (UID: \"9b00296d-a49b-4174-b08f-191b31d4c7a5\") " pod="kube-system/konnectivity-agent-dzlq8"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.770850    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d44eac64-207b-4b14-b10b-9d54cd3ae49b-kubelet-config\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771175    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5gcxt"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.770873    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-cnibin\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771512    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-run-netns\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771539    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ddafc9c5-84a7-432c-8907-41b4cbf2087a-multus-daemon-config\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771564    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-var-lib-cni-bin\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771590    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771615    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-multus-cni-dir\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-system-cni-dir\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771686    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-cnibin\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771716    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.773647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771744    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d44eac64-207b-4b14-b10b-9d54cd3ae49b-dbus\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-var-lib-kubelet\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771793    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771862    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgnxq\" (UniqueName: \"kubernetes.io/projected/311b2011-44ab-4733-a214-04809f2186f6-kube-api-access-jgnxq\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771886    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b00296d-a49b-4174-b08f-191b31d4c7a5-konnectivity-ca\") pod \"konnectivity-agent-dzlq8\" (UID: \"9b00296d-a49b-4174-b08f-191b31d4c7a5\") " pod="kube-system/konnectivity-agent-dzlq8"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771911    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-os-release\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771958    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-run-k8s-cni-cncf-io\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.771996    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-hostroot\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.772042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-system-cni-dir\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.772072    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ddafc9c5-84a7-432c-8907-41b4cbf2087a-cni-binary-copy\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.772101    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-var-lib-cni-multus\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.772128    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-run-multus-certs\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.772152    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcxn4\" (UniqueName: \"kubernetes.io/projected/ddafc9c5-84a7-432c-8907-41b4cbf2087a-kube-api-access-rcxn4\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.772176    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjbb7\" (UniqueName: \"kubernetes.io/projected/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-kube-api-access-zjbb7\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.772219    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-multus-socket-dir-parent\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.772251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-multus-conf-dir\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.775199 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.772274    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-etc-kubernetes\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.772296    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.772313    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcgtg\" (UniqueName: \"kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg\") pod \"network-check-target-h7n2n\" (UID: \"74033f51-c97e-44e8-81cc-87ed6a2a086e\") " pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.772708    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x5lfm"
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.773453    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.773519    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.773673    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hbxhk\""
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.773783    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.773871    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.774003    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.774464    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rvh4q\""
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.774660    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.775374    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.775496    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.775599    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 17:40:26.777277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.775622    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8qb8p\""
Jul 06 17:40:26.778182 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.777645    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 17:40:26.778182 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.777819    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 17:40:26.778182 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.777887    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 17:40:26.779810 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.778797    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 17:40:26.779810 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.779107    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 17:40:26.779810 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.779130    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 17:40:26.779810 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.779308    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k4blr\""
Jul 06 17:40:26.789938 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.786718    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tcssg"
Jul 06 17:40:26.793552 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.792483    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tcssg"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.855708    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.872819    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-sysconfig\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.872853    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-host\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.872881    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c643821-3178-4bab-9617-761f79b783b4-host-slash\") pod \"iptables-alerter-x5lfm\" (UID: \"8c643821-3178-4bab-9617-761f79b783b4\") " pod="openshift-network-operator/iptables-alerter-x5lfm"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.872923    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.872955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-multus-cni-dir\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.872978    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-node-log\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd-host\") pod \"node-ca-5gcxt\" (UID: \"ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd\") " pod="openshift-image-registry/node-ca-5gcxt"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873050    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-var-lib-openvswitch\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873075    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/90c1e591-8959-4588-9ce6-5b3058ffb43f-ovnkube-config\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873099    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-socket-dir\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873124    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-modprobe-d\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873147    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-var-lib-kubelet\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873170    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-run-netns\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873194    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-run-multus-certs\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.880582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873220    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcxn4\" (UniqueName: \"kubernetes.io/projected/ddafc9c5-84a7-432c-8907-41b4cbf2087a-kube-api-access-rcxn4\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873244    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a79ce496-9313-4d12-912a-b17d2429b3a6-tmp-dir\") pod \"node-resolver-v58n8\" (UID: \"a79ce496-9313-4d12-912a-b17d2429b3a6\") " pod="openshift-dns/node-resolver-v58n8"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873298    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-slash\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-multus-socket-dir-parent\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873349    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-etc-kubernetes\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873398    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-systemd\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873422    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-kubernetes\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873449    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-tuned\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873474    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsrmp\" (UniqueName: \"kubernetes.io/projected/ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd-kube-api-access-dsrmp\") pod \"node-ca-5gcxt\" (UID: \"ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd\") " pod="openshift-image-registry/node-ca-5gcxt"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xpr7\" (UniqueName: \"kubernetes.io/projected/8c643821-3178-4bab-9617-761f79b783b4-kube-api-access-5xpr7\") pod \"iptables-alerter-x5lfm\" (UID: \"8c643821-3178-4bab-9617-761f79b783b4\") " pod="openshift-network-operator/iptables-alerter-x5lfm"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873523    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/90c1e591-8959-4588-9ce6-5b3058ffb43f-ovn-node-metrics-cert\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873547    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48bm2\" (UniqueName: \"kubernetes.io/projected/90c1e591-8959-4588-9ce6-5b3058ffb43f-kube-api-access-48bm2\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ddafc9c5-84a7-432c-8907-41b4cbf2087a-multus-daemon-config\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873596    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-etc-selinux\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873624    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd-serviceca\") pod \"node-ca-5gcxt\" (UID: \"ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd\") " pod="openshift-image-registry/node-ca-5gcxt"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873649    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-run-systemd\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873673    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-run-openvswitch\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-cni-bin\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873747    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-var-lib-cni-bin\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873775    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-sysctl-conf\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873800    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-sys\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873824    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-etc-openvswitch\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-system-cni-dir\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-cnibin\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873915    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873949    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-device-dir\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.873991    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c643821-3178-4bab-9617-761f79b783b4-iptables-alerter-script\") pod \"iptables-alerter-x5lfm\" (UID: \"8c643821-3178-4bab-9617-761f79b783b4\") " pod="openshift-network-operator/iptables-alerter-x5lfm"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874020    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d44eac64-207b-4b14-b10b-9d54cd3ae49b-dbus\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-var-lib-kubelet\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874078    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874105    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgnxq\" (UniqueName: \"kubernetes.io/projected/311b2011-44ab-4733-a214-04809f2186f6-kube-api-access-jgnxq\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874134    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-kubelet\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.888710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874159    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-run-ovn\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874200    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-run-ovn-kubernetes\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874228    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b00296d-a49b-4174-b08f-191b31d4c7a5-konnectivity-ca\") pod \"konnectivity-agent-dzlq8\" (UID: \"9b00296d-a49b-4174-b08f-191b31d4c7a5\") " pod="kube-system/konnectivity-agent-dzlq8"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874256    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-os-release\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874283    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-run-k8s-cni-cncf-io\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-hostroot\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874335    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s4mp\" (UniqueName: \"kubernetes.io/projected/79ffdc21-54c7-4400-9a2f-ec713ec8a294-kube-api-access-2s4mp\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874364    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-systemd-units\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874393    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-system-cni-dir\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874422    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ddafc9c5-84a7-432c-8907-41b4cbf2087a-cni-binary-copy\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-var-lib-cni-multus\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjbb7\" (UniqueName: \"kubernetes.io/projected/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-kube-api-access-zjbb7\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874503    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kxdv\" (UniqueName: \"kubernetes.io/projected/a79ce496-9313-4d12-912a-b17d2429b3a6-kube-api-access-6kxdv\") pod \"node-resolver-v58n8\" (UID: \"a79ce496-9313-4d12-912a-b17d2429b3a6\") " pod="openshift-dns/node-resolver-v58n8"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874528    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79ffdc21-54c7-4400-9a2f-ec713ec8a294-tmp\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874553    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-log-socket\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874581    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-multus-conf-dir\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874609    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcgtg\" (UniqueName: \"kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg\") pod \"network-check-target-h7n2n\" (UID: \"74033f51-c97e-44e8-81cc-87ed6a2a086e\") " pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:26.892107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874636    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874663    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-cni-netd\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/90c1e591-8959-4588-9ce6-5b3058ffb43f-env-overrides\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874746    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/90c1e591-8959-4588-9ce6-5b3058ffb43f-ovnkube-script-lib\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874778    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874807    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-os-release\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874836    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-registration-dir\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2spqg\" (UniqueName: \"kubernetes.io/projected/873f883a-b711-44f7-b76c-a3d838f27467-kube-api-access-2spqg\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874892    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-sysctl-d\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874919    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b00296d-a49b-4174-b08f-191b31d4c7a5-agent-certs\") pod \"konnectivity-agent-dzlq8\" (UID: \"9b00296d-a49b-4174-b08f-191b31d4c7a5\") " pod="kube-system/konnectivity-agent-dzlq8"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d44eac64-207b-4b14-b10b-9d54cd3ae49b-kubelet-config\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.874994    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-cnibin\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.875023    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-run-netns\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.875050    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-sys-fs\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.875077    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a79ce496-9313-4d12-912a-b17d2429b3a6-hosts-file\") pod \"node-resolver-v58n8\" (UID: \"a79ce496-9313-4d12-912a-b17d2429b3a6\") " pod="openshift-dns/node-resolver-v58n8"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.875103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-run\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.875143    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-lib-modules\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.893250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.875170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.875758    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876081    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d44eac64-207b-4b14-b10b-9d54cd3ae49b-dbus\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876248    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-var-lib-kubelet\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876323    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-etc-kubernetes\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876394    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-run-multus-certs\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876570    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-multus-socket-dir-parent\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876588    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876658    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-var-lib-cni-bin\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876676    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-multus-cni-dir\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876722    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-cnibin\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876762    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ddafc9c5-84a7-432c-8907-41b4cbf2087a-multus-daemon-config\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:26.876836    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876867    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-system-cni-dir\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876868    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-multus-conf-dir\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:26.876895    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs podName:311b2011-44ab-4733-a214-04809f2186f6 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:27.376875542 +0000 UTC m=+2.970330057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs") pod "network-metrics-daemon-fsm9w" (UID: "311b2011-44ab-4733-a214-04809f2186f6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876916    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:26.876968    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 17:40:26.894672 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876984    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-os-release\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.876989    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-hostroot\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.877025    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d44eac64-207b-4b14-b10b-9d54cd3ae49b-kubelet-config\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:26.877035    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret podName:d44eac64-207b-4b14-b10b-9d54cd3ae49b nodeName:}" failed. No retries permitted until 2026-07-06 17:40:27.377017798 +0000 UTC m=+2.970472307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret") pod "global-pull-secret-syncer-vpff9" (UID: "d44eac64-207b-4b14-b10b-9d54cd3ae49b") : object "kube-system"/"original-pull-secret" not registered
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.877044    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-cnibin\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.877108    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-run-k8s-cni-cncf-io\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.877115    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-os-release\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.877169    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-system-cni-dir\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.877180    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.877197    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-var-lib-cni-multus\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.877205    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.877238    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ddafc9c5-84a7-432c-8907-41b4cbf2087a-host-run-netns\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.877486    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b00296d-a49b-4174-b08f-191b31d4c7a5-konnectivity-ca\") pod \"konnectivity-agent-dzlq8\" (UID: \"9b00296d-a49b-4174-b08f-191b31d4c7a5\") " pod="kube-system/konnectivity-agent-dzlq8"
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.877566    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ddafc9c5-84a7-432c-8907-41b4cbf2087a-cni-binary-copy\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.882503    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b00296d-a49b-4174-b08f-191b31d4c7a5-agent-certs\") pod \"konnectivity-agent-dzlq8\" (UID: \"9b00296d-a49b-4174-b08f-191b31d4c7a5\") " pod="kube-system/konnectivity-agent-dzlq8"
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:26.884717    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:26.884737    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 17:40:26.896335 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:26.884751    2581 projected.go:196] Error preparing data for projected volume kube-api-access-pcgtg for pod openshift-network-diagnostics/network-check-target-h7n2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 17:40:26.897453 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:26.884802    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg podName:74033f51-c97e-44e8-81cc-87ed6a2a086e nodeName:}" failed. No retries permitted until 2026-07-06 17:40:27.384785382 +0000 UTC m=+2.978239918 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pcgtg" (UniqueName: "kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg") pod "network-check-target-h7n2n" (UID: "74033f51-c97e-44e8-81cc-87ed6a2a086e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 17:40:26.897453 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.886722    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjbb7\" (UniqueName: \"kubernetes.io/projected/a4d538c1-5ceb-4481-84e8-6d5d48883f7f-kube-api-access-zjbb7\") pod \"multus-additional-cni-plugins-mfpln\" (UID: \"a4d538c1-5ceb-4481-84e8-6d5d48883f7f\") " pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:26.897453 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.887395    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgnxq\" (UniqueName: \"kubernetes.io/projected/311b2011-44ab-4733-a214-04809f2186f6-kube-api-access-jgnxq\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:26.897453 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.889552    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcxn4\" (UniqueName: \"kubernetes.io/projected/ddafc9c5-84a7-432c-8907-41b4cbf2087a-kube-api-access-rcxn4\") pod \"multus-4k5bv\" (UID: \"ddafc9c5-84a7-432c-8907-41b4cbf2087a\") " pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.910737    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal" event={"ID":"28add01306b01a8c01388eec6d6c37cf","Type":"ContainerStarted","Data":"0d3a72ff2f322b17f98f85f2efe7a538cd507960be7bcca9be90562f8c3a2218"}
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.912228    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-236.ec2.internal" event={"ID":"f393a703fb7ce0f536ef1eee45ebaae3","Type":"ContainerStarted","Data":"00fcdc0d840887feaca328f1987fcc853e099d2a0e74ddf99786b08e87050b13"}
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975616    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-sys-fs\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975652    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a79ce496-9313-4d12-912a-b17d2429b3a6-hosts-file\") pod \"node-resolver-v58n8\" (UID: \"a79ce496-9313-4d12-912a-b17d2429b3a6\") " pod="openshift-dns/node-resolver-v58n8"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-run\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975692    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-sys-fs\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975757    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a79ce496-9313-4d12-912a-b17d2429b3a6-hosts-file\") pod \"node-resolver-v58n8\" (UID: \"a79ce496-9313-4d12-912a-b17d2429b3a6\") " pod="openshift-dns/node-resolver-v58n8"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975701    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-lib-modules\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975789    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-run\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-sysconfig\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975814    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-lib-modules\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-host\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c643821-3178-4bab-9617-761f79b783b4-host-slash\") pod \"iptables-alerter-x5lfm\" (UID: \"8c643821-3178-4bab-9617-761f79b783b4\") " pod="openshift-network-operator/iptables-alerter-x5lfm"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975866    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-host\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975854    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-sysconfig\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975880    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975888    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c643821-3178-4bab-9617-761f79b783b4-host-slash\") pod \"iptables-alerter-x5lfm\" (UID: \"8c643821-3178-4bab-9617-761f79b783b4\") " pod="openshift-network-operator/iptables-alerter-x5lfm"
Jul 06 17:40:26.992285 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-node-log\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975935    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd-host\") pod \"node-ca-5gcxt\" (UID: \"ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd\") " pod="openshift-image-registry/node-ca-5gcxt"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975972    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-var-lib-openvswitch\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.975997    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/90c1e591-8959-4588-9ce6-5b3058ffb43f-ovnkube-config\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976032    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-socket-dir\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976054    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd-host\") pod \"node-ca-5gcxt\" (UID: \"ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd\") " pod="openshift-image-registry/node-ca-5gcxt"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976054    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-var-lib-openvswitch\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976104    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-node-log\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976062    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-modprobe-d\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976141    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-var-lib-kubelet\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976152    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-modprobe-d\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976167    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-run-netns\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976169    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-socket-dir\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976211    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-run-netns\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976220    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-var-lib-kubelet\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a79ce496-9313-4d12-912a-b17d2429b3a6-tmp-dir\") pod \"node-resolver-v58n8\" (UID: \"a79ce496-9313-4d12-912a-b17d2429b3a6\") " pod="openshift-dns/node-resolver-v58n8"
Jul 06 17:40:26.993705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976266    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-slash\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-systemd\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-kubernetes\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976340    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-tuned\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976367    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsrmp\" (UniqueName: \"kubernetes.io/projected/ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd-kube-api-access-dsrmp\") pod \"node-ca-5gcxt\" (UID: \"ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd\") " pod="openshift-image-registry/node-ca-5gcxt"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xpr7\" (UniqueName: \"kubernetes.io/projected/8c643821-3178-4bab-9617-761f79b783b4-kube-api-access-5xpr7\") pod \"iptables-alerter-x5lfm\" (UID: \"8c643821-3178-4bab-9617-761f79b783b4\") " pod="openshift-network-operator/iptables-alerter-x5lfm"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/90c1e591-8959-4588-9ce6-5b3058ffb43f-ovn-node-metrics-cert\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976460    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-kubernetes\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48bm2\" (UniqueName: \"kubernetes.io/projected/90c1e591-8959-4588-9ce6-5b3058ffb43f-kube-api-access-48bm2\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976514    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-etc-selinux\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976540    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a79ce496-9313-4d12-912a-b17d2429b3a6-tmp-dir\") pod \"node-resolver-v58n8\" (UID: \"a79ce496-9313-4d12-912a-b17d2429b3a6\") " pod="openshift-dns/node-resolver-v58n8"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd-serviceca\") pod \"node-ca-5gcxt\" (UID: \"ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd\") " pod="openshift-image-registry/node-ca-5gcxt"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976592    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-systemd\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976601    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-run-systemd\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976411    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-slash\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-run-openvswitch\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-cni-bin\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.995875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976704    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-sysctl-conf\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976724    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-etc-selinux\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976750    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-cni-bin\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976767    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-run-openvswitch\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976797    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-sys\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976824    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-etc-openvswitch\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976853    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/90c1e591-8959-4588-9ce6-5b3058ffb43f-ovnkube-config\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976869    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-device-dir\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976909    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c643821-3178-4bab-9617-761f79b783b4-iptables-alerter-script\") pod \"iptables-alerter-x5lfm\" (UID: \"8c643821-3178-4bab-9617-761f79b783b4\") " pod="openshift-network-operator/iptables-alerter-x5lfm"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976971    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-run-systemd\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976964    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-sysctl-conf\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976910    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-etc-openvswitch\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.976969    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-sys\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977048    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-device-dir\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-kubelet\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977093    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-kubelet\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-run-ovn\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977122    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd-serviceca\") pod \"node-ca-5gcxt\" (UID: \"ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd\") " pod="openshift-image-registry/node-ca-5gcxt"
Jul 06 17:40:26.996842 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977132    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-run-ovn-kubernetes\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977127    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-run-ovn\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977171    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-run-ovn-kubernetes\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977180    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s4mp\" (UniqueName: \"kubernetes.io/projected/79ffdc21-54c7-4400-9a2f-ec713ec8a294-kube-api-access-2s4mp\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977210    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-systemd-units\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kxdv\" (UniqueName: \"kubernetes.io/projected/a79ce496-9313-4d12-912a-b17d2429b3a6-kube-api-access-6kxdv\") pod \"node-resolver-v58n8\" (UID: \"a79ce496-9313-4d12-912a-b17d2429b3a6\") " pod="openshift-dns/node-resolver-v58n8"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977268    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79ffdc21-54c7-4400-9a2f-ec713ec8a294-tmp\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977294    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-log-socket\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977309    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-systemd-units\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977334    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-cni-netd\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977386    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/90c1e591-8959-4588-9ce6-5b3058ffb43f-env-overrides\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977415    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/90c1e591-8959-4588-9ce6-5b3058ffb43f-ovnkube-script-lib\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977442    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977455    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-registration-dir\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2spqg\" (UniqueName: \"kubernetes.io/projected/873f883a-b711-44f7-b76c-a3d838f27467-kube-api-access-2spqg\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977509    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-sysctl-d\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:26.999465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977514    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-log-socket\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977442    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c643821-3178-4bab-9617-761f79b783b4-iptables-alerter-script\") pod \"iptables-alerter-x5lfm\" (UID: \"8c643821-3178-4bab-9617-761f79b783b4\") " pod="openshift-network-operator/iptables-alerter-x5lfm"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977484    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/90c1e591-8959-4588-9ce6-5b3058ffb43f-host-cni-netd\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977597    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/873f883a-b711-44f7-b76c-a3d838f27467-registration-dir\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.977627    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-sysctl-d\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.978125    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/90c1e591-8959-4588-9ce6-5b3058ffb43f-env-overrides\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.978167    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/90c1e591-8959-4588-9ce6-5b3058ffb43f-ovnkube-script-lib\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.979803    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/90c1e591-8959-4588-9ce6-5b3058ffb43f-ovn-node-metrics-cert\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.979983    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79ffdc21-54c7-4400-9a2f-ec713ec8a294-tmp\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.980158    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/79ffdc21-54c7-4400-9a2f-ec713ec8a294-etc-tuned\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.988483    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2spqg\" (UniqueName: \"kubernetes.io/projected/873f883a-b711-44f7-b76c-a3d838f27467-kube-api-access-2spqg\") pod \"aws-ebs-csi-driver-node-xm9d2\" (UID: \"873f883a-b711-44f7-b76c-a3d838f27467\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.988506    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsrmp\" (UniqueName: \"kubernetes.io/projected/ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd-kube-api-access-dsrmp\") pod \"node-ca-5gcxt\" (UID: \"ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd\") " pod="openshift-image-registry/node-ca-5gcxt"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.989497    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s4mp\" (UniqueName: \"kubernetes.io/projected/79ffdc21-54c7-4400-9a2f-ec713ec8a294-kube-api-access-2s4mp\") pod \"tuned-fmknh\" (UID: \"79ffdc21-54c7-4400-9a2f-ec713ec8a294\") " pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.990016    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48bm2\" (UniqueName: \"kubernetes.io/projected/90c1e591-8959-4588-9ce6-5b3058ffb43f-kube-api-access-48bm2\") pod \"ovnkube-node-xqg68\" (UID: \"90c1e591-8959-4588-9ce6-5b3058ffb43f\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.990178    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xpr7\" (UniqueName: \"kubernetes.io/projected/8c643821-3178-4bab-9617-761f79b783b4-kube-api-access-5xpr7\") pod \"iptables-alerter-x5lfm\" (UID: \"8c643821-3178-4bab-9617-761f79b783b4\") " pod="openshift-network-operator/iptables-alerter-x5lfm"
Jul 06 17:40:27.000451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:26.993690    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kxdv\" (UniqueName: \"kubernetes.io/projected/a79ce496-9313-4d12-912a-b17d2429b3a6-kube-api-access-6kxdv\") pod \"node-resolver-v58n8\" (UID: \"a79ce496-9313-4d12-912a-b17d2429b3a6\") " pod="openshift-dns/node-resolver-v58n8"
Jul 06 17:40:27.087273 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.062688    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dzlq8"
Jul 06 17:40:27.087273 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.078616    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4k5bv"
Jul 06 17:40:27.157356 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.088366    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mfpln"
Jul 06 17:40:27.157356 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.091179    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2"
Jul 06 17:40:27.157356 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:27.097199    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b00296d_a49b_4174_b08f_191b31d4c7a5.slice/crio-87b4ffb3908629cff311eea941e3d0dd7ca1f2ed7a75f91feeb3fcfd38e4f64a WatchSource:0}: Error finding container 87b4ffb3908629cff311eea941e3d0dd7ca1f2ed7a75f91feeb3fcfd38e4f64a: Status 404 returned error can't find the container with id 87b4ffb3908629cff311eea941e3d0dd7ca1f2ed7a75f91feeb3fcfd38e4f64a
Jul 06 17:40:27.157356 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:27.097440    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddafc9c5_84a7_432c_8907_41b4cbf2087a.slice/crio-1731df4d6071eb83218056c39161664de168b2697d4b1c145493551ec162f171 WatchSource:0}: Error finding container 1731df4d6071eb83218056c39161664de168b2697d4b1c145493551ec162f171: Status 404 returned error can't find the container with id 1731df4d6071eb83218056c39161664de168b2697d4b1c145493551ec162f171
Jul 06 17:40:27.157356 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.106032    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v58n8"
Jul 06 17:40:27.157356 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.114574    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fmknh"
Jul 06 17:40:27.157356 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.122777    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5gcxt"
Jul 06 17:40:27.157356 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.131059    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x5lfm"
Jul 06 17:40:27.157356 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.137466    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:27.157356 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.157055    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 17:40:27.186696 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:40:27.186624    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae64e4d4_c8c5_47aa_92d6_3cdb8a2a6edd.slice/crio-32fd0b5a5e9cdaac54c68eaefd8e96f6869b61a27bfbccd86b939aa779aa3130 WatchSource:0}: Error finding container 32fd0b5a5e9cdaac54c68eaefd8e96f6869b61a27bfbccd86b939aa779aa3130: Status 404 returned error can't find the container with id 32fd0b5a5e9cdaac54c68eaefd8e96f6869b61a27bfbccd86b939aa779aa3130
Jul 06 17:40:27.380492 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.380456    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:27.380666 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.380523    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:27.380666 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:27.380611    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 17:40:27.380771 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:27.380663    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 17:40:27.380771 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:27.380672    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs podName:311b2011-44ab-4733-a214-04809f2186f6 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:28.380653455 +0000 UTC m=+3.974107977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs") pod "network-metrics-daemon-fsm9w" (UID: "311b2011-44ab-4733-a214-04809f2186f6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 17:40:27.380771 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:27.380738    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret podName:d44eac64-207b-4b14-b10b-9d54cd3ae49b nodeName:}" failed. No retries permitted until 2026-07-06 17:40:28.380717221 +0000 UTC m=+3.974171737 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret") pod "global-pull-secret-syncer-vpff9" (UID: "d44eac64-207b-4b14-b10b-9d54cd3ae49b") : object "kube-system"/"original-pull-secret" not registered
Jul 06 17:40:27.481913 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.481840    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcgtg\" (UniqueName: \"kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg\") pod \"network-check-target-h7n2n\" (UID: \"74033f51-c97e-44e8-81cc-87ed6a2a086e\") " pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:27.482086 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:27.482004    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 17:40:27.482086 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:27.482020    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 17:40:27.482086 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:27.482029    2581 projected.go:196] Error preparing data for projected volume kube-api-access-pcgtg for pod openshift-network-diagnostics/network-check-target-h7n2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 17:40:27.482086 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:27.482077    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg podName:74033f51-c97e-44e8-81cc-87ed6a2a086e nodeName:}" failed. No retries permitted until 2026-07-06 17:40:28.482062643 +0000 UTC m=+4.075517153 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pcgtg" (UniqueName: "kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg") pod "network-check-target-h7n2n" (UID: "74033f51-c97e-44e8-81cc-87ed6a2a086e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 17:40:27.794519 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.794416    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 17:35:27 +0000 UTC" deadline="2028-01-04 21:06:08.323308015 +0000 UTC"
Jul 06 17:40:27.794519 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.794477    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13131h25m40.528835361s"
Jul 06 17:40:27.906593 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.906559    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:27.906774 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:27.906706    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:27.915823 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.915786    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dzlq8" event={"ID":"9b00296d-a49b-4174-b08f-191b31d4c7a5","Type":"ContainerStarted","Data":"87b4ffb3908629cff311eea941e3d0dd7ca1f2ed7a75f91feeb3fcfd38e4f64a"}
Jul 06 17:40:27.917036 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.917007    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5gcxt" event={"ID":"ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd","Type":"ContainerStarted","Data":"32fd0b5a5e9cdaac54c68eaefd8e96f6869b61a27bfbccd86b939aa779aa3130"}
Jul 06 17:40:27.918279 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.918248    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fmknh" event={"ID":"79ffdc21-54c7-4400-9a2f-ec713ec8a294","Type":"ContainerStarted","Data":"0efa97f0a0940e4f64aaf8c981995a0ac2561740bd2da67eb10cd07abc3d181c"}
Jul 06 17:40:27.919512 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.919481    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x5lfm" event={"ID":"8c643821-3178-4bab-9617-761f79b783b4","Type":"ContainerStarted","Data":"c604d9d986d133de057ddef68b9a47ffd096d086d9af3bf88ac58d99535f192d"}
Jul 06 17:40:27.920749 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.920724    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v58n8" event={"ID":"a79ce496-9313-4d12-912a-b17d2429b3a6","Type":"ContainerStarted","Data":"0ea2de64ce5b39a11762e80344a0f4f4965f63971e9b1f15b75b9677b22cc89a"}
Jul 06 17:40:27.921970 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.921947    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2" event={"ID":"873f883a-b711-44f7-b76c-a3d838f27467","Type":"ContainerStarted","Data":"55dc678860bc1c9438e8814e187198d1437af6e6f5e151db0d262b52e7ae7c5f"}
Jul 06 17:40:27.923135 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.923106    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpln" event={"ID":"a4d538c1-5ceb-4481-84e8-6d5d48883f7f","Type":"ContainerStarted","Data":"3a218ec77f575c7e2b98da38f25f09c1d675abcc13fbf7d8bd5d4238e3d991e9"}
Jul 06 17:40:27.924384 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.924363    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68" event={"ID":"90c1e591-8959-4588-9ce6-5b3058ffb43f","Type":"ContainerStarted","Data":"c1aa93d121f6d2cbf0547052b16ed09679fb81f91141f2a4e6fda05e111522e7"}
Jul 06 17:40:27.925568 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:27.925547    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4k5bv" event={"ID":"ddafc9c5-84a7-432c-8907-41b4cbf2087a","Type":"ContainerStarted","Data":"1731df4d6071eb83218056c39161664de168b2697d4b1c145493551ec162f171"}
Jul 06 17:40:28.392281 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:28.392245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:28.393005 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:28.392369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:28.393005 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:28.392425    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 17:40:28.393005 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:28.392496    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret podName:d44eac64-207b-4b14-b10b-9d54cd3ae49b nodeName:}" failed. No retries permitted until 2026-07-06 17:40:30.39247676 +0000 UTC m=+5.985931284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret") pod "global-pull-secret-syncer-vpff9" (UID: "d44eac64-207b-4b14-b10b-9d54cd3ae49b") : object "kube-system"/"original-pull-secret" not registered
Jul 06 17:40:28.393005 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:28.392512    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 17:40:28.393005 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:28.392566    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs podName:311b2011-44ab-4733-a214-04809f2186f6 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:30.392548364 +0000 UTC m=+5.986002876 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs") pod "network-metrics-daemon-fsm9w" (UID: "311b2011-44ab-4733-a214-04809f2186f6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 17:40:28.493363 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:28.493321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcgtg\" (UniqueName: \"kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg\") pod \"network-check-target-h7n2n\" (UID: \"74033f51-c97e-44e8-81cc-87ed6a2a086e\") " pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:28.493635 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:28.493608    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 17:40:28.493712 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:28.493642    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 17:40:28.493712 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:28.493656    2581 projected.go:196] Error preparing data for projected volume kube-api-access-pcgtg for pod openshift-network-diagnostics/network-check-target-h7n2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 17:40:28.493827 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:28.493719    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg podName:74033f51-c97e-44e8-81cc-87ed6a2a086e nodeName:}" failed. No retries permitted until 2026-07-06 17:40:30.493698033 +0000 UTC m=+6.087152551 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pcgtg" (UniqueName: "kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg") pod "network-check-target-h7n2n" (UID: "74033f51-c97e-44e8-81cc-87ed6a2a086e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 17:40:28.795175 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:28.795072    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 17:35:27 +0000 UTC" deadline="2028-04-04 17:13:10.402408976 +0000 UTC"
Jul 06 17:40:28.795175 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:28.795111    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15311h32m41.60730247s"
Jul 06 17:40:28.908000 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:28.906698    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:28.908000 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:28.906829    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:28.908000 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:28.907282    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:28.908000 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:28.907374    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:29.906632 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:29.906586    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:29.907122 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:29.906749    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:30.407457 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:30.407426    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:30.407641 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:30.407513    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:30.407704 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:30.407643    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 17:40:30.407756 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:30.407702    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs podName:311b2011-44ab-4733-a214-04809f2186f6 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:34.407683651 +0000 UTC m=+10.001138174 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs") pod "network-metrics-daemon-fsm9w" (UID: "311b2011-44ab-4733-a214-04809f2186f6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 17:40:30.408129 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:30.408109    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 17:40:30.408216 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:30.408168    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret podName:d44eac64-207b-4b14-b10b-9d54cd3ae49b nodeName:}" failed. No retries permitted until 2026-07-06 17:40:34.408151919 +0000 UTC m=+10.001606431 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret") pod "global-pull-secret-syncer-vpff9" (UID: "d44eac64-207b-4b14-b10b-9d54cd3ae49b") : object "kube-system"/"original-pull-secret" not registered
Jul 06 17:40:30.508308 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:30.508270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcgtg\" (UniqueName: \"kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg\") pod \"network-check-target-h7n2n\" (UID: \"74033f51-c97e-44e8-81cc-87ed6a2a086e\") " pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:30.508486 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:30.508470    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 17:40:30.508543 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:30.508498    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 17:40:30.508543 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:30.508514    2581 projected.go:196] Error preparing data for projected volume kube-api-access-pcgtg for pod openshift-network-diagnostics/network-check-target-h7n2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 17:40:30.508668 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:30.508578    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg podName:74033f51-c97e-44e8-81cc-87ed6a2a086e nodeName:}" failed. No retries permitted until 2026-07-06 17:40:34.508554002 +0000 UTC m=+10.102008534 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pcgtg" (UniqueName: "kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg") pod "network-check-target-h7n2n" (UID: "74033f51-c97e-44e8-81cc-87ed6a2a086e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 17:40:30.913132 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:30.912635    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:30.913132 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:30.912781    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:30.913132 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:30.912794    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:30.913132 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:30.912866    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:31.906095 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:31.905993    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:31.906252 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:31.906127    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:32.907670 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:32.906936    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:32.907670 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:32.907055    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:32.907670 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:32.907471    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:32.907670 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:32.907565    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:33.906238 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:33.906172    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:33.906408 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:33.906307    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:34.438278 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:34.438248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:34.438706 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:34.438297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:34.438706 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:34.438412    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 17:40:34.438706 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:34.438415    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 17:40:34.438706 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:34.438473    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret podName:d44eac64-207b-4b14-b10b-9d54cd3ae49b nodeName:}" failed. No retries permitted until 2026-07-06 17:40:42.438453364 +0000 UTC m=+18.031907920 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret") pod "global-pull-secret-syncer-vpff9" (UID: "d44eac64-207b-4b14-b10b-9d54cd3ae49b") : object "kube-system"/"original-pull-secret" not registered
Jul 06 17:40:34.438706 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:34.438493    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs podName:311b2011-44ab-4733-a214-04809f2186f6 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:42.438482871 +0000 UTC m=+18.031937381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs") pod "network-metrics-daemon-fsm9w" (UID: "311b2011-44ab-4733-a214-04809f2186f6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 17:40:34.539633 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:34.539601    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcgtg\" (UniqueName: \"kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg\") pod \"network-check-target-h7n2n\" (UID: \"74033f51-c97e-44e8-81cc-87ed6a2a086e\") " pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:34.539795 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:34.539777    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 17:40:34.539866 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:34.539801    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 17:40:34.539866 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:34.539812    2581 projected.go:196] Error preparing data for projected volume kube-api-access-pcgtg for pod openshift-network-diagnostics/network-check-target-h7n2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 17:40:34.540027 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:34.539871    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg podName:74033f51-c97e-44e8-81cc-87ed6a2a086e nodeName:}" failed. No retries permitted until 2026-07-06 17:40:42.539852594 +0000 UTC m=+18.133307104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pcgtg" (UniqueName: "kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg") pod "network-check-target-h7n2n" (UID: "74033f51-c97e-44e8-81cc-87ed6a2a086e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 17:40:34.909053 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:34.908980    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:34.909181 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:34.909080    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:34.909181 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:34.909146    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:34.909274 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:34.909233    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:35.907239 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:35.906978    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:35.907677 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:35.907348    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:36.909801 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:36.909770    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:36.910245 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:36.909882    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:36.910245 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:36.909970    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:36.910245 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:36.910078    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:37.906472 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:37.906448    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:37.906627 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:37.906606    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:37.955432 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:37.955387    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-236.ec2.internal" event={"ID":"f393a703fb7ce0f536ef1eee45ebaae3","Type":"ContainerStarted","Data":"7583c879508bc496221c4f5ef9efd8dcfa6c36a156e92b624a8158740c53d5ff"}
Jul 06 17:40:37.957524 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:37.957470    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fmknh" event={"ID":"79ffdc21-54c7-4400-9a2f-ec713ec8a294","Type":"ContainerStarted","Data":"eb4eee32637a78955320af30f14773e4ab65e3e8a7ab8c604b424caabe3583e5"}
Jul 06 17:40:37.969594 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:37.969556    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-236.ec2.internal" podStartSLOduration=11.969545117 podStartE2EDuration="11.969545117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:37.969277361 +0000 UTC m=+13.562731891" watchObservedRunningTime="2026-07-06 17:40:37.969545117 +0000 UTC m=+13.562999646"
Jul 06 17:40:37.984183 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:37.984149    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fmknh" podStartSLOduration=2.6099826630000003 podStartE2EDuration="12.984136544s" totalImagesPullingTime="10.374153881s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:37.983849884 +0000 UTC m=+13.577304414" watchObservedRunningTime="2026-07-06 17:40:37.984136544 +0000 UTC m=+13.577591076"
Jul 06 17:40:38.906444 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:38.906252    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:38.906564 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:38.906309    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:38.906564 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:38.906534    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:38.906677 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:38.906629    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:38.961148 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:38.961120    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2" event={"ID":"873f883a-b711-44f7-b76c-a3d838f27467","Type":"ContainerStarted","Data":"a9743fd3bb2c5635ba890346a5cf6310c84d44a9276b900c28e226e0e84f0b0d"}
Jul 06 17:40:38.962907 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:38.962852    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpln" event={"ID":"a4d538c1-5ceb-4481-84e8-6d5d48883f7f","Type":"ContainerStarted","Data":"900eade90d957bfc5cdd9bb3dafcd5b159c029e1d405aa9919cb01f82b4e0727"}
Jul 06 17:40:38.964453 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:38.964427    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dzlq8" event={"ID":"9b00296d-a49b-4174-b08f-191b31d4c7a5","Type":"ContainerStarted","Data":"c7dd2ec96a4e14333a23095043fd8f9b3be7d357f10bec7dde76cc3ce1692770"}
Jul 06 17:40:38.966170 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:38.966125    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal" event={"ID":"28add01306b01a8c01388eec6d6c37cf","Type":"ContainerStarted","Data":"fd0a2bc7e4431e86208e8e2e5417aaec6d092c9aec4442b89c542e4f52a3cccb"}
Jul 06 17:40:38.967636 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:38.967612    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5gcxt" event={"ID":"ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd","Type":"ContainerStarted","Data":"808bc90c99ca33b623ea56997c7e9ca0012f4e8ac0027d280f79638c17050c7e"}
Jul 06 17:40:38.969296 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:38.969273    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v58n8" event={"ID":"a79ce496-9313-4d12-912a-b17d2429b3a6","Type":"ContainerStarted","Data":"d8dfcc01c034573ef137de9831be6a6a26269abf9fd5bd3b04340ec9043d5b71"}
Jul 06 17:40:38.998080 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:38.998025    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-v58n8" podStartSLOduration=3.641892945 podStartE2EDuration="13.998008268s" totalImagesPullingTime="10.356115323s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:38.997988333 +0000 UTC m=+14.591442865" watchObservedRunningTime="2026-07-06 17:40:38.998008268 +0000 UTC m=+14.591462800"
Jul 06 17:40:39.011854 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:39.011814    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dzlq8" podStartSLOduration=3.85599696 podStartE2EDuration="14.011797941s" totalImagesPullingTime="10.155800981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:39.01133923 +0000 UTC m=+14.604793763" watchObservedRunningTime="2026-07-06 17:40:39.011797941 +0000 UTC m=+14.605252475"
Jul 06 17:40:39.024272 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:39.024199    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5gcxt" podStartSLOduration=3.957213302 podStartE2EDuration="14.024182651s" totalImagesPullingTime="10.066969349s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:39.023959618 +0000 UTC m=+14.617414148" watchObservedRunningTime="2026-07-06 17:40:39.024182651 +0000 UTC m=+14.617637183"
Jul 06 17:40:39.043756 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:39.043735    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dzlq8"
Jul 06 17:40:39.044332 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:39.044313    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dzlq8"
Jul 06 17:40:39.906154 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:39.906124    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:39.906327 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:39.906228    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:39.972013 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:39.971989    2581 generic.go:356] "Generic (PLEG): container finished" podID="28add01306b01a8c01388eec6d6c37cf" containerID="fd0a2bc7e4431e86208e8e2e5417aaec6d092c9aec4442b89c542e4f52a3cccb" exitCode=0
Jul 06 17:40:39.972453 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:39.972074    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal" event={"ID":"28add01306b01a8c01388eec6d6c37cf","Type":"ContainerDied","Data":"fd0a2bc7e4431e86208e8e2e5417aaec6d092c9aec4442b89c542e4f52a3cccb"}
Jul 06 17:40:39.973611 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:39.973584    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x5lfm" event={"ID":"8c643821-3178-4bab-9617-761f79b783b4","Type":"ContainerStarted","Data":"578c543c0b240b34cc9165b4fbf147565db8063f10451d8e0486247aebcf0e59"}
Jul 06 17:40:39.975073 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:39.975051    2581 generic.go:356] "Generic (PLEG): container finished" podID="a4d538c1-5ceb-4481-84e8-6d5d48883f7f" containerID="900eade90d957bfc5cdd9bb3dafcd5b159c029e1d405aa9919cb01f82b4e0727" exitCode=0
Jul 06 17:40:39.975149 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:39.975127    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpln" event={"ID":"a4d538c1-5ceb-4481-84e8-6d5d48883f7f","Type":"ContainerDied","Data":"900eade90d957bfc5cdd9bb3dafcd5b159c029e1d405aa9919cb01f82b4e0727"}
Jul 06 17:40:40.089880 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:40.089856    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 17:40:40.856154 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:40.856023    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T17:40:40.089878866Z","UUID":"f45a3a09-82db-48c9-9827-408778933c18","Handler":null,"Name":"","Endpoint":""}
Jul 06 17:40:40.858107 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:40.858081    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 17:40:40.858252 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:40.858127    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 17:40:40.910704 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:40.910678    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:40.910841 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:40.910787    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:40.911162 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:40.911143    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:40.911261 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:40.911236    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:40.980882 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:40.980856    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal" event={"ID":"28add01306b01a8c01388eec6d6c37cf","Type":"ContainerStarted","Data":"723827d184ef3141b136524f00f1f490176855b85e2c6831f9f3e14191bbfeb6"}
Jul 06 17:40:40.984543 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:40.983652    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2" event={"ID":"873f883a-b711-44f7-b76c-a3d838f27467","Type":"ContainerStarted","Data":"9804be13556a56be712924cda2c54676723ee2b0f2e13ef45d3fc7e1cda6dfbe"}
Jul 06 17:40:40.984543 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:40.983780    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 06 17:40:40.995371 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:40.995208    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x5lfm" podStartSLOduration=5.638589301 podStartE2EDuration="15.995195128s" totalImagesPullingTime="10.356605827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:40.021063947 +0000 UTC m=+15.614518475" watchObservedRunningTime="2026-07-06 17:40:40.995195128 +0000 UTC m=+16.588649664"
Jul 06 17:40:41.906216 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:41.906184    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:41.906413 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:41.906332    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:42.499971 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:42.499938    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:42.500436 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:42.499998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:42.500436 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:42.500106    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 17:40:42.500436 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:42.500171    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret podName:d44eac64-207b-4b14-b10b-9d54cd3ae49b nodeName:}" failed. No retries permitted until 2026-07-06 17:40:58.500157053 +0000 UTC m=+34.093611561 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret") pod "global-pull-secret-syncer-vpff9" (UID: "d44eac64-207b-4b14-b10b-9d54cd3ae49b") : object "kube-system"/"original-pull-secret" not registered
Jul 06 17:40:42.500436 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:42.500106    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 17:40:42.500436 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:42.500240    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs podName:311b2011-44ab-4733-a214-04809f2186f6 nodeName:}" failed. No retries permitted until 2026-07-06 17:40:58.500228974 +0000 UTC m=+34.093683497 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs") pod "network-metrics-daemon-fsm9w" (UID: "311b2011-44ab-4733-a214-04809f2186f6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 17:40:42.600528 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:42.600287    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcgtg\" (UniqueName: \"kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg\") pod \"network-check-target-h7n2n\" (UID: \"74033f51-c97e-44e8-81cc-87ed6a2a086e\") " pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:42.600712 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:42.600460    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 17:40:42.600712 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:42.600631    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 17:40:42.600712 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:42.600647    2581 projected.go:196] Error preparing data for projected volume kube-api-access-pcgtg for pod openshift-network-diagnostics/network-check-target-h7n2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 17:40:42.600712 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:42.600706    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg podName:74033f51-c97e-44e8-81cc-87ed6a2a086e nodeName:}" failed. No retries permitted until 2026-07-06 17:40:58.600688702 +0000 UTC m=+34.194143214 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pcgtg" (UniqueName: "kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg") pod "network-check-target-h7n2n" (UID: "74033f51-c97e-44e8-81cc-87ed6a2a086e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 17:40:42.910084 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:42.909979    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:42.910241 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:42.910091    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:42.910241 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:42.910154    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:42.910312 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:42.910259    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:43.906041 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:43.905998    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:43.906520 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:43.906124    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:44.907148 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:44.907115    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:44.907564 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:44.907213    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:44.907564 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:44.907259    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:44.907564 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:44.907336    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:45.906647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:45.906610    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:45.906804 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:45.906754    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:46.906072 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:46.906031    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:46.906072 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:46.906059    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:46.906685 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:46.906160    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:46.906685 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:46.906283    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:47.906565 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:47.906533    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:47.907045 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:47.906664    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:48.906551 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:48.906521    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:48.906828 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:48.906660    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:48.906828 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:48.906695    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:48.906828 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:48.906789    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:49.906089 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:49.906053    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:49.906255 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:49.906194    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:50.906267 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:50.906234    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:50.906632 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:50.906327    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:50.906632 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:50.906386    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:50.906632 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:50.906472    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:51.906876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:51.906848    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:51.907382 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:51.906974    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:52.013528 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.013487    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2" event={"ID":"873f883a-b711-44f7-b76c-a3d838f27467","Type":"ContainerStarted","Data":"24be0d7ed154738b94b8ff1d5a14d4b72fdbaef2198dd1468bdeb5fc6bd18963"}
Jul 06 17:40:52.015583 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.015554    2581 generic.go:356] "Generic (PLEG): container finished" podID="a4d538c1-5ceb-4481-84e8-6d5d48883f7f" containerID="6aa33c44f13554de4c43503bbc785fd31d28b49e9bf50d924abdbc482daa1876" exitCode=0
Jul 06 17:40:52.015756 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.015633    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpln" event={"ID":"a4d538c1-5ceb-4481-84e8-6d5d48883f7f","Type":"ContainerDied","Data":"6aa33c44f13554de4c43503bbc785fd31d28b49e9bf50d924abdbc482daa1876"}
Jul 06 17:40:52.019280 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.019256    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68" event={"ID":"90c1e591-8959-4588-9ce6-5b3058ffb43f","Type":"ContainerStarted","Data":"b1cae848a871613cd1f59cdd9cbed1d475a1813e502e1c8f47dd950cc78ab11e"}
Jul 06 17:40:52.019404 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.019319    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68" event={"ID":"90c1e591-8959-4588-9ce6-5b3058ffb43f","Type":"ContainerStarted","Data":"3cece003bff7788cf011f85841e647d9aa356d74a25cdb3f4cfc533f2ff7ecfb"}
Jul 06 17:40:52.019453 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.019420    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68" event={"ID":"90c1e591-8959-4588-9ce6-5b3058ffb43f","Type":"ContainerStarted","Data":"617649bf41d52a03e1ef30e9fffe84c1e738c714051646cabefb945cfd21f5ef"}
Jul 06 17:40:52.019493 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.019471    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68" event={"ID":"90c1e591-8959-4588-9ce6-5b3058ffb43f","Type":"ContainerStarted","Data":"d40bbc42ccd190efb35a50b0b318bcf064a5323a4627a0bc0dcec3811ad2de2e"}
Jul 06 17:40:52.019534 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.019508    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68" event={"ID":"90c1e591-8959-4588-9ce6-5b3058ffb43f","Type":"ContainerStarted","Data":"4360196f84607a0502d3a21a4c93dc7dd95f91a5b482f32d2bc7342f91ff5de4"}
Jul 06 17:40:52.019576 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.019538    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68" event={"ID":"90c1e591-8959-4588-9ce6-5b3058ffb43f","Type":"ContainerStarted","Data":"6f66f9a7bcdaa1d425c9deac3844ba4634b40f62a2a77c8933b1981c88d1de66"}
Jul 06 17:40:52.020827 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.020805    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4k5bv" event={"ID":"ddafc9c5-84a7-432c-8907-41b4cbf2087a","Type":"ContainerStarted","Data":"52c89f661ab308834e67cffec725e9a65648a061b21c9585259c1ebc189a0973"}
Jul 06 17:40:52.032963 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.032919    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-236.ec2.internal" podStartSLOduration=26.032889758 podStartE2EDuration="26.032889758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:40.994846424 +0000 UTC m=+16.588300954" watchObservedRunningTime="2026-07-06 17:40:52.032889758 +0000 UTC m=+27.626344287"
Jul 06 17:40:52.033223 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.033203    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xm9d2" podStartSLOduration=4.648836755 podStartE2EDuration="27.033198351s" totalImagesPullingTime="22.384361596s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:52.032598527 +0000 UTC m=+27.626053057" watchObservedRunningTime="2026-07-06 17:40:52.033198351 +0000 UTC m=+27.626652878"
Jul 06 17:40:52.050108 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.050073    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4k5bv" podStartSLOduration=3.125495741 podStartE2EDuration="27.050060963s" totalImagesPullingTime="23.924565222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:52.049684849 +0000 UTC m=+27.643139380" watchObservedRunningTime="2026-07-06 17:40:52.050060963 +0000 UTC m=+27.643515493"
Jul 06 17:40:52.906918 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.906880    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:52.907461 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:52.906880    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:52.907461 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:52.907005    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:52.907461 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:52.907072    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:53.877951 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:53.877930    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dzlq8"
Jul 06 17:40:53.878099 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:53.878029    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 06 17:40:53.878496 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:53.878477    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dzlq8"
Jul 06 17:40:53.906235 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:53.906214    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:53.906340 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:53.906307    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:54.029654 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:54.029601    2581 generic.go:356] "Generic (PLEG): container finished" podID="a4d538c1-5ceb-4481-84e8-6d5d48883f7f" containerID="aa2769f3f34daddd10544df97400846c656789ad96482b5f6c021325edd6f61d" exitCode=0
Jul 06 17:40:54.030117 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:54.029666    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpln" event={"ID":"a4d538c1-5ceb-4481-84e8-6d5d48883f7f","Type":"ContainerDied","Data":"aa2769f3f34daddd10544df97400846c656789ad96482b5f6c021325edd6f61d"}
Jul 06 17:40:54.033017 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:54.032989    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68" event={"ID":"90c1e591-8959-4588-9ce6-5b3058ffb43f","Type":"ContainerStarted","Data":"08a55b57dfd787746423799db149b5dfa16abec804ddc7dee5e30a4f57fa4378"}
Jul 06 17:40:54.907161 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:54.907131    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:54.907312 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:54.907207    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:54.907361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:54.907311    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:54.907448 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:54.907425    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:55.906711 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:55.906669    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:55.907087 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:55.906811    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:56.039471 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:56.039163    2581 generic.go:356] "Generic (PLEG): container finished" podID="a4d538c1-5ceb-4481-84e8-6d5d48883f7f" containerID="c522a496d105468ccdd4089dfa60e7c44cd31212e4d4e0c14686647755abd3a3" exitCode=0
Jul 06 17:40:56.039471 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:56.039244    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpln" event={"ID":"a4d538c1-5ceb-4481-84e8-6d5d48883f7f","Type":"ContainerDied","Data":"c522a496d105468ccdd4089dfa60e7c44cd31212e4d4e0c14686647755abd3a3"}
Jul 06 17:40:56.044300 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:56.043861    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68" event={"ID":"90c1e591-8959-4588-9ce6-5b3058ffb43f","Type":"ContainerStarted","Data":"4744acaea651fedf27c3a04e11adee04de3abde801685ac5e11d9f643181e711"}
Jul 06 17:40:56.044300 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:56.044144    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:56.044300 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:56.044206    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:56.061992 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:56.061968    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:56.095428 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:56.095387    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68" podStartSLOduration=7.202115911 podStartE2EDuration="31.09537463s" totalImagesPullingTime="23.893258719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:40:56.094034171 +0000 UTC m=+31.687488702" watchObservedRunningTime="2026-07-06 17:40:56.09537463 +0000 UTC m=+31.688829159"
Jul 06 17:40:56.906682 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:56.906655    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:56.906848 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:56.906740    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:56.907219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:56.906850    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:56.907219 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:56.906970    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:57.046432 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:57.046406    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:57.064016 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:57.063997    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:40:57.906519 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:57.906496    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:57.906697 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:57.906604    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:57.942061 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:57.942035    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vpff9"]
Jul 06 17:40:57.942413 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:57.942110    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:57.942413 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:57.942176    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:57.947671 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:57.947648    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fsm9w"]
Jul 06 17:40:57.948251 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:57.948228    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h7n2n"]
Jul 06 17:40:57.948329 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:57.948301    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:57.948411 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:57.948387    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:58.047649 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:58.047630    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:58.047725 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:58.047709    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:40:58.523149 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:58.521936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:58.523149 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:58.522029    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:58.523149 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:58.522175    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 17:40:58.523149 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:58.522246    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs podName:311b2011-44ab-4733-a214-04809f2186f6 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:30.522226108 +0000 UTC m=+66.115680620 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs") pod "network-metrics-daemon-fsm9w" (UID: "311b2011-44ab-4733-a214-04809f2186f6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 17:40:58.523149 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:58.522701    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 17:40:58.523149 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:58.522755    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret podName:d44eac64-207b-4b14-b10b-9d54cd3ae49b nodeName:}" failed. No retries permitted until 2026-07-06 17:41:30.522740631 +0000 UTC m=+66.116195144 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret") pod "global-pull-secret-syncer-vpff9" (UID: "d44eac64-207b-4b14-b10b-9d54cd3ae49b") : object "kube-system"/"original-pull-secret" not registered
Jul 06 17:40:58.622722 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:58.622693    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcgtg\" (UniqueName: \"kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg\") pod \"network-check-target-h7n2n\" (UID: \"74033f51-c97e-44e8-81cc-87ed6a2a086e\") " pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:58.622858 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:58.622843    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 17:40:58.622930 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:58.622862    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 17:40:58.622930 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:58.622871    2581 projected.go:196] Error preparing data for projected volume kube-api-access-pcgtg for pod openshift-network-diagnostics/network-check-target-h7n2n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 17:40:58.622930 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:58.622924    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg podName:74033f51-c97e-44e8-81cc-87ed6a2a086e nodeName:}" failed. No retries permitted until 2026-07-06 17:41:30.622913038 +0000 UTC m=+66.216367549 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pcgtg" (UniqueName: "kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg") pod "network-check-target-h7n2n" (UID: "74033f51-c97e-44e8-81cc-87ed6a2a086e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 17:40:59.905912 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:59.905877    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:40:59.906345 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:59.905879    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:40:59.906345 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:59.906009    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:40:59.906345 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:40:59.906042    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:40:59.906345 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:59.906148    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:40:59.906345 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:40:59.906251    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:41:01.906708 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:01.906507    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:41:01.907153 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:01.906529    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:41:01.907153 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:01.906796    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:41:01.907153 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:01.906568    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:41:01.907153 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:01.906864    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h7n2n" podUID="74033f51-c97e-44e8-81cc-87ed6a2a086e"
Jul 06 17:41:01.907153 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:01.906999    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpff9" podUID="d44eac64-207b-4b14-b10b-9d54cd3ae49b"
Jul 06 17:41:03.697686 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.697659    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-236.ec2.internal" event="NodeReady"
Jul 06 17:41:03.698153 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.697758    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 17:41:03.731271 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.731141    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"]
Jul 06 17:41:03.745025 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.744275    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"]
Jul 06 17:41:03.745025 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.744710    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.749210 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.748143    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 17:41:03.749210 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.748227    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 06 17:41:03.749210 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.748376    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 06 17:41:03.749210 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.748489    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 06 17:41:03.749210 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.748811    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 06 17:41:03.749210 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.749090    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 17:41:03.749210 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.749129    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 17:41:03.758138 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.758120    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-767d584684-w8wk7"]
Jul 06 17:41:03.758253 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.758237    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:41:03.760646 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.760629    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 06 17:41:03.760727 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.760702    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 06 17:41:03.760795 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.760767    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xsgc2\""
Jul 06 17:41:03.772239 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.772223    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc"]
Jul 06 17:41:03.772329 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.772321    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.776123 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.776098    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v256q\""
Jul 06 17:41:03.776123 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.776120    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 17:41:03.776297 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.776207    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 17:41:03.776356 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.776310    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 17:41:03.781429 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.781412    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 17:41:03.792398 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.792382    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"]
Jul 06 17:41:03.792494 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.792479    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc"
Jul 06 17:41:03.795271 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.795130    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-tsnrv\""
Jul 06 17:41:03.795580 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.795555    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 06 17:41:03.816306 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.816289    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"]
Jul 06 17:41:03.816391 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.816313    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc"]
Jul 06 17:41:03.816391 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.816323    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"]
Jul 06 17:41:03.816391 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.816335    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"]
Jul 06 17:41:03.816391 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.816347    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-767d584684-w8wk7"]
Jul 06 17:41:03.816391 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.816359    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xnt6h"]
Jul 06 17:41:03.816391 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.816385    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:41:03.818691 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.818678    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 06 17:41:03.832387 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.832371    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-97mtx"]
Jul 06 17:41:03.832486 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.832472    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xnt6h"
Jul 06 17:41:03.835595 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.835285    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 17:41:03.835595 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.835364    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 17:41:03.835595 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.835408    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 17:41:03.835595 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.835288    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xxwrm\""
Jul 06 17:41:03.847039 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.847021    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xnt6h"]
Jul 06 17:41:03.847105 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.847047    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-97mtx"]
Jul 06 17:41:03.847105 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.847099    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:03.849298 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.849278    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 17:41:03.849389 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.849348    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 17:41:03.849389 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.849354    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lmt7g\""
Jul 06 17:41:03.863414 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d99db248-8fac-4e30-acd8-661343bb2ff2-installation-pull-secrets\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.863492 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863419    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79rdh\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-kube-api-access-79rdh\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.863492 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863447    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-certificates\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.863492 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863465    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/798f1d53-692a-4d26-be8c-6118ebf9b4c6-hub\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.863492 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863480    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-bound-sa-token\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.863645 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863533    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f3682cbe-9ac5-4915-9e80-8c897e4a3996-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:41:03.863645 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863565    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d99db248-8fac-4e30-acd8-661343bb2ff2-ca-trust-extracted\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.863645 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863587    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/798f1d53-692a-4d26-be8c-6118ebf9b4c6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.863645 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863604    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:41:03.863645 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863623    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d99db248-8fac-4e30-acd8-661343bb2ff2-trusted-ca\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.863826 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863656    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s4d5\" (UniqueName: \"kubernetes.io/projected/798f1d53-692a-4d26-be8c-6118ebf9b4c6-kube-api-access-9s4d5\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.863826 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863676    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.863826 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863741    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/798f1d53-692a-4d26-be8c-6118ebf9b4c6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.863826 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863767    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/798f1d53-692a-4d26-be8c-6118ebf9b4c6-ca\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.863826 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863799    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/798f1d53-692a-4d26-be8c-6118ebf9b4c6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.863998 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.863833    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d99db248-8fac-4e30-acd8-661343bb2ff2-image-registry-private-configuration\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.906535 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.906517    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:41:03.906695 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.906682    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:41:03.906819 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.906787    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:41:03.908993 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.908964    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vsjgr\""
Jul 06 17:41:03.908993 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.908984    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gwzng\""
Jul 06 17:41:03.909142 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.908969    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 17:41:03.909142 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.909061    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 17:41:03.909142 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.909093    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 17:41:03.909292 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.909274    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 17:41:03.965044 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d99db248-8fac-4e30-acd8-661343bb2ff2-image-registry-private-configuration\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.965044 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965035    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3fe6180-c3f7-4b6f-bb58-591305c33c45-tmp-dir\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:03.965191 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-certificates\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.965191 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965072    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/798f1d53-692a-4d26-be8c-6118ebf9b4c6-hub\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.965191 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965088    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-bound-sa-token\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.965191 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965113    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxbq5\" (UniqueName: \"kubernetes.io/projected/a3fe6180-c3f7-4b6f-bb58-591305c33c45-kube-api-access-wxbq5\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:03.965191 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b6542aba-0caa-4b27-873b-7a46ae3df205-klusterlet-config\") pod \"klusterlet-addon-workmgr-79b9d59d7b-7c2kp\" (UID: \"b6542aba-0caa-4b27-873b-7a46ae3df205\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:41:03.965402 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d99db248-8fac-4e30-acd8-661343bb2ff2-trusted-ca\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.965402 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s4d5\" (UniqueName: \"kubernetes.io/projected/798f1d53-692a-4d26-be8c-6118ebf9b4c6-kube-api-access-9s4d5\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.965402 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965273    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.965402 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965339    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/798f1d53-692a-4d26-be8c-6118ebf9b4c6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.965402 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/798f1d53-692a-4d26-be8c-6118ebf9b4c6-ca\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.965402 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965399    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8rrd\" (UniqueName: \"kubernetes.io/projected/df45d8cf-5c5e-4e51-bc0a-029de6595c53-kube-api-access-w8rrd\") pod \"ingress-canary-xnt6h\" (UID: \"df45d8cf-5c5e-4e51-bc0a-029de6595c53\") " pod="openshift-ingress-canary/ingress-canary-xnt6h"
Jul 06 17:41:03.965651 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965428    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50fa955c-ef41-4b07-ae5a-04c8919d4dd7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7f568d4746-lh6zc\" (UID: \"50fa955c-ef41-4b07-ae5a-04c8919d4dd7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc"
Jul 06 17:41:03.965651 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:03.965457    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 17:41:03.965651 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965459    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkrm5\" (UniqueName: \"kubernetes.io/projected/50fa955c-ef41-4b07-ae5a-04c8919d4dd7-kube-api-access-qkrm5\") pod \"managed-serviceaccount-addon-agent-7f568d4746-lh6zc\" (UID: \"50fa955c-ef41-4b07-ae5a-04c8919d4dd7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc"
Jul 06 17:41:03.965651 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:03.965470    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-767d584684-w8wk7: secret "image-registry-tls" not found
Jul 06 17:41:03.965651 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965487    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3fe6180-c3f7-4b6f-bb58-591305c33c45-config-volume\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:03.965651 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:03.965543    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls podName:d99db248-8fac-4e30-acd8-661343bb2ff2 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:04.465509292 +0000 UTC m=+40.058963818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls") pod "image-registry-767d584684-w8wk7" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2") : secret "image-registry-tls" not found
Jul 06 17:41:03.965651 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965560    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79rdh\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-kube-api-access-79rdh\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.965651 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965617    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d99db248-8fac-4e30-acd8-661343bb2ff2-installation-pull-secrets\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.966030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965659    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:03.966030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f3682cbe-9ac5-4915-9e80-8c897e4a3996-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:41:03.966030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965732    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d99db248-8fac-4e30-acd8-661343bb2ff2-ca-trust-extracted\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.966030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965755    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-certificates\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.966030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965777    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/798f1d53-692a-4d26-be8c-6118ebf9b4c6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.966030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:41:03.966030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965864    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggssj\" (UniqueName: \"kubernetes.io/projected/b6542aba-0caa-4b27-873b-7a46ae3df205-kube-api-access-ggssj\") pod \"klusterlet-addon-workmgr-79b9d59d7b-7c2kp\" (UID: \"b6542aba-0caa-4b27-873b-7a46ae3df205\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:41:03.966030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965946    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6542aba-0caa-4b27-873b-7a46ae3df205-tmp\") pod \"klusterlet-addon-workmgr-79b9d59d7b-7c2kp\" (UID: \"b6542aba-0caa-4b27-873b-7a46ae3df205\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:41:03.966030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965975    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/798f1d53-692a-4d26-be8c-6118ebf9b4c6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.966030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.965996    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/df45d8cf-5c5e-4e51-bc0a-029de6595c53-cert\") pod \"ingress-canary-xnt6h\" (UID: \"df45d8cf-5c5e-4e51-bc0a-029de6595c53\") " pod="openshift-ingress-canary/ingress-canary-xnt6h"
Jul 06 17:41:03.966424 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.966176    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d99db248-8fac-4e30-acd8-661343bb2ff2-trusted-ca\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.966472 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.966452    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d99db248-8fac-4e30-acd8-661343bb2ff2-ca-trust-extracted\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.966615 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:03.966599    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 17:41:03.966665 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:03.966658    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert podName:f3682cbe-9ac5-4915-9e80-8c897e4a3996 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:04.466642761 +0000 UTC m=+40.060097283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-7lvhg" (UID: "f3682cbe-9ac5-4915-9e80-8c897e4a3996") : secret "networking-console-plugin-cert" not found
Jul 06 17:41:03.967519 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.967494    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f3682cbe-9ac5-4915-9e80-8c897e4a3996-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:41:03.967606 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.967534    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/798f1d53-692a-4d26-be8c-6118ebf9b4c6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.969589 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.969565    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/798f1d53-692a-4d26-be8c-6118ebf9b4c6-ca\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.969671 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.969613    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/798f1d53-692a-4d26-be8c-6118ebf9b4c6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.969671 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.969612    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d99db248-8fac-4e30-acd8-661343bb2ff2-image-registry-private-configuration\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.969746 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.969704    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/798f1d53-692a-4d26-be8c-6118ebf9b4c6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.969810 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.969790    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/798f1d53-692a-4d26-be8c-6118ebf9b4c6-hub\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:03.969810 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.969804    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d99db248-8fac-4e30-acd8-661343bb2ff2-installation-pull-secrets\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.973343 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.973321    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79rdh\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-kube-api-access-79rdh\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.973414 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.973324    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-bound-sa-token\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:03.973711 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:03.973695    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s4d5\" (UniqueName: \"kubernetes.io/projected/798f1d53-692a-4d26-be8c-6118ebf9b4c6-kube-api-access-9s4d5\") pod \"cluster-proxy-proxy-agent-7d567cfbc-ckjcr\" (UID: \"798f1d53-692a-4d26-be8c-6118ebf9b4c6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:04.062512 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.062487    2581 generic.go:356] "Generic (PLEG): container finished" podID="a4d538c1-5ceb-4481-84e8-6d5d48883f7f" containerID="0406a0d806805a08558be3262d972663a48be83a0dae94cd14005ba9228b9b53" exitCode=0
Jul 06 17:41:04.062576 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.062533    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpln" event={"ID":"a4d538c1-5ceb-4481-84e8-6d5d48883f7f","Type":"ContainerDied","Data":"0406a0d806805a08558be3262d972663a48be83a0dae94cd14005ba9228b9b53"}
Jul 06 17:41:04.066447 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.066428    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkrm5\" (UniqueName: \"kubernetes.io/projected/50fa955c-ef41-4b07-ae5a-04c8919d4dd7-kube-api-access-qkrm5\") pod \"managed-serviceaccount-addon-agent-7f568d4746-lh6zc\" (UID: \"50fa955c-ef41-4b07-ae5a-04c8919d4dd7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc"
Jul 06 17:41:04.066557 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.066454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3fe6180-c3f7-4b6f-bb58-591305c33c45-config-volume\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:04.066557 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.066488    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:04.066866 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.066668    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggssj\" (UniqueName: \"kubernetes.io/projected/b6542aba-0caa-4b27-873b-7a46ae3df205-kube-api-access-ggssj\") pod \"klusterlet-addon-workmgr-79b9d59d7b-7c2kp\" (UID: \"b6542aba-0caa-4b27-873b-7a46ae3df205\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:41:04.066866 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.066712    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6542aba-0caa-4b27-873b-7a46ae3df205-tmp\") pod \"klusterlet-addon-workmgr-79b9d59d7b-7c2kp\" (UID: \"b6542aba-0caa-4b27-873b-7a46ae3df205\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:41:04.066866 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.066736    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:41:04.066866 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:04.066773    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 17:41:04.066866 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:04.066826    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls podName:a3fe6180-c3f7-4b6f-bb58-591305c33c45 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:04.566809944 +0000 UTC m=+40.160264452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls") pod "dns-default-97mtx" (UID: "a3fe6180-c3f7-4b6f-bb58-591305c33c45") : secret "dns-default-metrics-tls" not found
Jul 06 17:41:04.066866 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.066857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/df45d8cf-5c5e-4e51-bc0a-029de6595c53-cert\") pod \"ingress-canary-xnt6h\" (UID: \"df45d8cf-5c5e-4e51-bc0a-029de6595c53\") " pod="openshift-ingress-canary/ingress-canary-xnt6h"
Jul 06 17:41:04.067233 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.066941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3fe6180-c3f7-4b6f-bb58-591305c33c45-tmp-dir\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:04.067233 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.066964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxbq5\" (UniqueName: \"kubernetes.io/projected/a3fe6180-c3f7-4b6f-bb58-591305c33c45-kube-api-access-wxbq5\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:04.067233 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.066981    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b6542aba-0caa-4b27-873b-7a46ae3df205-klusterlet-config\") pod \"klusterlet-addon-workmgr-79b9d59d7b-7c2kp\" (UID: \"b6542aba-0caa-4b27-873b-7a46ae3df205\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:41:04.067233 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.067019    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8rrd\" (UniqueName: \"kubernetes.io/projected/df45d8cf-5c5e-4e51-bc0a-029de6595c53-kube-api-access-w8rrd\") pod \"ingress-canary-xnt6h\" (UID: \"df45d8cf-5c5e-4e51-bc0a-029de6595c53\") " pod="openshift-ingress-canary/ingress-canary-xnt6h"
Jul 06 17:41:04.067233 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.067036    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50fa955c-ef41-4b07-ae5a-04c8919d4dd7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7f568d4746-lh6zc\" (UID: \"50fa955c-ef41-4b07-ae5a-04c8919d4dd7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc"
Jul 06 17:41:04.067233 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.067064    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3fe6180-c3f7-4b6f-bb58-591305c33c45-config-volume\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:04.067233 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.067066    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6542aba-0caa-4b27-873b-7a46ae3df205-tmp\") pod \"klusterlet-addon-workmgr-79b9d59d7b-7c2kp\" (UID: \"b6542aba-0caa-4b27-873b-7a46ae3df205\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:41:04.067577 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.067332    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3fe6180-c3f7-4b6f-bb58-591305c33c45-tmp-dir\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:04.069088 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.069069    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50fa955c-ef41-4b07-ae5a-04c8919d4dd7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7f568d4746-lh6zc\" (UID: \"50fa955c-ef41-4b07-ae5a-04c8919d4dd7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc"
Jul 06 17:41:04.069474 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.069456    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/df45d8cf-5c5e-4e51-bc0a-029de6595c53-cert\") pod \"ingress-canary-xnt6h\" (UID: \"df45d8cf-5c5e-4e51-bc0a-029de6595c53\") " pod="openshift-ingress-canary/ingress-canary-xnt6h"
Jul 06 17:41:04.069559 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.069480    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b6542aba-0caa-4b27-873b-7a46ae3df205-klusterlet-config\") pod \"klusterlet-addon-workmgr-79b9d59d7b-7c2kp\" (UID: \"b6542aba-0caa-4b27-873b-7a46ae3df205\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:41:04.075403 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.075385    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkrm5\" (UniqueName: \"kubernetes.io/projected/50fa955c-ef41-4b07-ae5a-04c8919d4dd7-kube-api-access-qkrm5\") pod \"managed-serviceaccount-addon-agent-7f568d4746-lh6zc\" (UID: \"50fa955c-ef41-4b07-ae5a-04c8919d4dd7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc"
Jul 06 17:41:04.075821 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.075799    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxbq5\" (UniqueName: \"kubernetes.io/projected/a3fe6180-c3f7-4b6f-bb58-591305c33c45-kube-api-access-wxbq5\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:04.075949 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.075935    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggssj\" (UniqueName: \"kubernetes.io/projected/b6542aba-0caa-4b27-873b-7a46ae3df205-kube-api-access-ggssj\") pod \"klusterlet-addon-workmgr-79b9d59d7b-7c2kp\" (UID: \"b6542aba-0caa-4b27-873b-7a46ae3df205\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:41:04.076062 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.076048    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8rrd\" (UniqueName: \"kubernetes.io/projected/df45d8cf-5c5e-4e51-bc0a-029de6595c53-kube-api-access-w8rrd\") pod \"ingress-canary-xnt6h\" (UID: \"df45d8cf-5c5e-4e51-bc0a-029de6595c53\") " pod="openshift-ingress-canary/ingress-canary-xnt6h"
Jul 06 17:41:04.102986 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.102967    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc"
Jul 06 17:41:04.138651 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.138622    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:41:04.143682 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.143660    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xnt6h"
Jul 06 17:41:04.269867 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.269586    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"]
Jul 06 17:41:04.291478 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.291452    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc"]
Jul 06 17:41:04.313944 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.313921    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"]
Jul 06 17:41:04.325032 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.325011    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xnt6h"]
Jul 06 17:41:04.327722 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:41:04.327699    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf45d8cf_5c5e_4e51_bc0a_029de6595c53.slice/crio-321291ac5ca936a54aabe804027b8e097fb63636e4c72c7743c9da68188002c7 WatchSource:0}: Error finding container 321291ac5ca936a54aabe804027b8e097fb63636e4c72c7743c9da68188002c7: Status 404 returned error can't find the container with id 321291ac5ca936a54aabe804027b8e097fb63636e4c72c7743c9da68188002c7
Jul 06 17:41:04.328692 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:41:04.328658    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod798f1d53_692a_4d26_be8c_6118ebf9b4c6.slice/crio-c2ed84f583102effd739e07cc2c24ee1aa504de624a57b5649c408c7cb5867a2 WatchSource:0}: Error finding container c2ed84f583102effd739e07cc2c24ee1aa504de624a57b5649c408c7cb5867a2: Status 404 returned error can't find the container with id c2ed84f583102effd739e07cc2c24ee1aa504de624a57b5649c408c7cb5867a2
Jul 06 17:41:04.329881 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:41:04.329634    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6542aba_0caa_4b27_873b_7a46ae3df205.slice/crio-37c54910b16e4e2847d9bd7eec2044b8398f2819eccb8e6ed639137ba64c1916 WatchSource:0}: Error finding container 37c54910b16e4e2847d9bd7eec2044b8398f2819eccb8e6ed639137ba64c1916: Status 404 returned error can't find the container with id 37c54910b16e4e2847d9bd7eec2044b8398f2819eccb8e6ed639137ba64c1916
Jul 06 17:41:04.332511 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:41:04.332464    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50fa955c_ef41_4b07_ae5a_04c8919d4dd7.slice/crio-63858a422dc5e20b14fa3fad3c1a4915d558357d4410019ca9418a0c4ad819ac WatchSource:0}: Error finding container 63858a422dc5e20b14fa3fad3c1a4915d558357d4410019ca9418a0c4ad819ac: Status 404 returned error can't find the container with id 63858a422dc5e20b14fa3fad3c1a4915d558357d4410019ca9418a0c4ad819ac
Jul 06 17:41:04.471705 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.471680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:04.471841 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.471745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:41:04.471841 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:04.471824    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 17:41:04.471841 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:04.471834    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 17:41:04.471841 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:04.471838    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-767d584684-w8wk7: secret "image-registry-tls" not found
Jul 06 17:41:04.472006 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:04.471875    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert podName:f3682cbe-9ac5-4915-9e80-8c897e4a3996 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:05.471863509 +0000 UTC m=+41.065318016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-7lvhg" (UID: "f3682cbe-9ac5-4915-9e80-8c897e4a3996") : secret "networking-console-plugin-cert" not found
Jul 06 17:41:04.472006 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:04.471888    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls podName:d99db248-8fac-4e30-acd8-661343bb2ff2 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:05.471881533 +0000 UTC m=+41.065336040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls") pod "image-registry-767d584684-w8wk7" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2") : secret "image-registry-tls" not found
Jul 06 17:41:04.572194 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:04.572130    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:04.572323 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:04.572298    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 17:41:04.572385 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:04.572373    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls podName:a3fe6180-c3f7-4b6f-bb58-591305c33c45 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:05.572351696 +0000 UTC m=+41.165806206 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls") pod "dns-default-97mtx" (UID: "a3fe6180-c3f7-4b6f-bb58-591305c33c45") : secret "dns-default-metrics-tls" not found
Jul 06 17:41:05.069542 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:05.069506    2581 generic.go:356] "Generic (PLEG): container finished" podID="a4d538c1-5ceb-4481-84e8-6d5d48883f7f" containerID="4ab700af1f3e2a39ab4cda1aea7577c59a3c14b673ebb8ddad1640ea962908ba" exitCode=0
Jul 06 17:41:05.070262 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:05.069589    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpln" event={"ID":"a4d538c1-5ceb-4481-84e8-6d5d48883f7f","Type":"ContainerDied","Data":"4ab700af1f3e2a39ab4cda1aea7577c59a3c14b673ebb8ddad1640ea962908ba"}
Jul 06 17:41:05.071174 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:05.071147    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc" event={"ID":"50fa955c-ef41-4b07-ae5a-04c8919d4dd7","Type":"ContainerStarted","Data":"63858a422dc5e20b14fa3fad3c1a4915d558357d4410019ca9418a0c4ad819ac"}
Jul 06 17:41:05.073186 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:05.073163    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xnt6h" event={"ID":"df45d8cf-5c5e-4e51-bc0a-029de6595c53","Type":"ContainerStarted","Data":"321291ac5ca936a54aabe804027b8e097fb63636e4c72c7743c9da68188002c7"}
Jul 06 17:41:05.076762 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:05.076737    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp" event={"ID":"b6542aba-0caa-4b27-873b-7a46ae3df205","Type":"ContainerStarted","Data":"37c54910b16e4e2847d9bd7eec2044b8398f2819eccb8e6ed639137ba64c1916"}
Jul 06 17:41:05.078250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:05.078227    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr" event={"ID":"798f1d53-692a-4d26-be8c-6118ebf9b4c6","Type":"ContainerStarted","Data":"c2ed84f583102effd739e07cc2c24ee1aa504de624a57b5649c408c7cb5867a2"}
Jul 06 17:41:05.480517 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:05.480285    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:05.480694 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:05.480436    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 17:41:05.480694 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:05.480600    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-767d584684-w8wk7: secret "image-registry-tls" not found
Jul 06 17:41:05.480694 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:05.480620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:41:05.480694 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:05.480663    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls podName:d99db248-8fac-4e30-acd8-661343bb2ff2 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:07.480643189 +0000 UTC m=+43.074097701 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls") pod "image-registry-767d584684-w8wk7" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2") : secret "image-registry-tls" not found
Jul 06 17:41:05.480937 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:05.480832    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 17:41:05.480937 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:05.480886    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert podName:f3682cbe-9ac5-4915-9e80-8c897e4a3996 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:07.480873065 +0000 UTC m=+43.074327576 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-7lvhg" (UID: "f3682cbe-9ac5-4915-9e80-8c897e4a3996") : secret "networking-console-plugin-cert" not found
Jul 06 17:41:05.581699 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:05.581656    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:05.581868 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:05.581796    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 17:41:05.581954 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:05.581860    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls podName:a3fe6180-c3f7-4b6f-bb58-591305c33c45 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:07.581839604 +0000 UTC m=+43.175294128 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls") pod "dns-default-97mtx" (UID: "a3fe6180-c3f7-4b6f-bb58-591305c33c45") : secret "dns-default-metrics-tls" not found
Jul 06 17:41:06.098167 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:06.098126    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mfpln" event={"ID":"a4d538c1-5ceb-4481-84e8-6d5d48883f7f","Type":"ContainerStarted","Data":"2e580dde0b47e12a79e041bc04da372c4305431bc1d3055d975a4482f1497c6a"}
Jul 06 17:41:06.121033 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:06.120974    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mfpln" podStartSLOduration=10.206042977 podStartE2EDuration="41.12095357s" totalImagesPullingTime="30.914910593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:06.118991951 +0000 UTC m=+41.712446482" watchObservedRunningTime="2026-07-06 17:41:06.12095357 +0000 UTC m=+41.714408102"
Jul 06 17:41:07.499760 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:07.499600    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:07.499760 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:07.499738    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:41:07.500234 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:07.499763    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 17:41:07.500234 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:07.499784    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-767d584684-w8wk7: secret "image-registry-tls" not found
Jul 06 17:41:07.500234 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:07.499851    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls podName:d99db248-8fac-4e30-acd8-661343bb2ff2 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:11.499827132 +0000 UTC m=+47.093281666 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls") pod "image-registry-767d584684-w8wk7" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2") : secret "image-registry-tls" not found
Jul 06 17:41:07.500234 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:07.499915    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 17:41:07.500234 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:07.499985    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert podName:f3682cbe-9ac5-4915-9e80-8c897e4a3996 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:11.499968839 +0000 UTC m=+47.093423349 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-7lvhg" (UID: "f3682cbe-9ac5-4915-9e80-8c897e4a3996") : secret "networking-console-plugin-cert" not found
Jul 06 17:41:07.601083 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:07.601053    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:07.601232 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:07.601184    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 17:41:07.601386 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:07.601272    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls podName:a3fe6180-c3f7-4b6f-bb58-591305c33c45 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:11.601251552 +0000 UTC m=+47.194706065 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls") pod "dns-default-97mtx" (UID: "a3fe6180-c3f7-4b6f-bb58-591305c33c45") : secret "dns-default-metrics-tls" not found
Jul 06 17:41:11.531292 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:11.531255    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:41:11.531732 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:11.531327    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:11.531732 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:11.531410    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 17:41:11.531732 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:11.531466    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 17:41:11.531732 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:11.531473    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert podName:f3682cbe-9ac5-4915-9e80-8c897e4a3996 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:19.531457071 +0000 UTC m=+55.124911579 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-7lvhg" (UID: "f3682cbe-9ac5-4915-9e80-8c897e4a3996") : secret "networking-console-plugin-cert" not found
Jul 06 17:41:11.531732 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:11.531480    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-767d584684-w8wk7: secret "image-registry-tls" not found
Jul 06 17:41:11.531732 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:11.531527    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls podName:d99db248-8fac-4e30-acd8-661343bb2ff2 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:19.531511998 +0000 UTC m=+55.124966526 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls") pod "image-registry-767d584684-w8wk7" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2") : secret "image-registry-tls" not found
Jul 06 17:41:11.631600 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:11.631572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:11.631715 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:11.631700    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 17:41:11.631760 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:11.631751    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls podName:a3fe6180-c3f7-4b6f-bb58-591305c33c45 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:19.631736239 +0000 UTC m=+55.225190757 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls") pod "dns-default-97mtx" (UID: "a3fe6180-c3f7-4b6f-bb58-591305c33c45") : secret "dns-default-metrics-tls" not found
Jul 06 17:41:12.114960 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:12.114928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xnt6h" event={"ID":"df45d8cf-5c5e-4e51-bc0a-029de6595c53","Type":"ContainerStarted","Data":"b39cb82c4c9617f73536c1378cc90a188bbbd7f88f1e513758c38461d53b4289"}
Jul 06 17:41:12.116611 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:12.116584    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp" event={"ID":"b6542aba-0caa-4b27-873b-7a46ae3df205","Type":"ContainerStarted","Data":"0424a9b974c126c52a9555d3f01be60cff6b7ccf1fb47c26946548a12353b2d9"}
Jul 06 17:41:12.118544 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:12.118523    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr" event={"ID":"798f1d53-692a-4d26-be8c-6118ebf9b4c6","Type":"ContainerStarted","Data":"e589372f25c3ce6749e7117898076460ce68bca8347b112a2e4b5ca57eb12d33"}
Jul 06 17:41:12.118619 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:12.118558    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr" event={"ID":"798f1d53-692a-4d26-be8c-6118ebf9b4c6","Type":"ContainerStarted","Data":"180d36bc08420c30f0192d7d7136a9950f1e8dc59485f71e666c1b6f22ec9cde"}
Jul 06 17:41:12.120054 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:12.120034    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc" event={"ID":"50fa955c-ef41-4b07-ae5a-04c8919d4dd7","Type":"ContainerStarted","Data":"34b0f5b3e5bacfe14842c5fe0ea2adefcba4f3476c70731e463a29e872fedeab"}
Jul 06 17:41:12.145395 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:12.145353    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xnt6h" podStartSLOduration=1.6468716730000001 podStartE2EDuration="9.145338577s" totalImagesPullingTime="7.498466904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:12.143302257 +0000 UTC m=+47.736756788" watchObservedRunningTime="2026-07-06 17:41:12.145338577 +0000 UTC m=+47.738793107"
Jul 06 17:41:12.156909 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:12.156851    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc" podStartSLOduration=2.660852782 podStartE2EDuration="10.156836029s" totalImagesPullingTime="7.495983247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:41:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:12.156777369 +0000 UTC m=+47.750231902" watchObservedRunningTime="2026-07-06 17:41:12.156836029 +0000 UTC m=+47.750290560"
Jul 06 17:41:12.175235 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:12.175201    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp" podStartSLOduration=2.66305561 podStartE2EDuration="10.175186548s" totalImagesPullingTime="7.512130938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:41:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:12.175066487 +0000 UTC m=+47.768521017" watchObservedRunningTime="2026-07-06 17:41:12.175186548 +0000 UTC m=+47.768641081"
Jul 06 17:41:13.125162 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:13.125126    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr" event={"ID":"798f1d53-692a-4d26-be8c-6118ebf9b4c6","Type":"ContainerStarted","Data":"5d716c03fb12e949b152ba20ae25cd1f5a63309e8a84bf2d7947180d079bb4bc"}
Jul 06 17:41:13.125605 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:13.125476    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:41:13.126982 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:13.126959    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:41:13.143426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:13.143377    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr" podStartSLOduration=3.637847754 podStartE2EDuration="11.143365052s" totalImagesPullingTime="7.505517298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:41:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:13.14170636 +0000 UTC m=+48.735160890" watchObservedRunningTime="2026-07-06 17:41:13.143365052 +0000 UTC m=+48.736819559"
Jul 06 17:41:19.596681 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:19.596644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:19.597155 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:19.596784    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:41:19.597155 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:19.596793    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 17:41:19.597155 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:19.596814    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-767d584684-w8wk7: secret "image-registry-tls" not found
Jul 06 17:41:19.597155 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:19.596870    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls podName:d99db248-8fac-4e30-acd8-661343bb2ff2 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:35.59685358 +0000 UTC m=+71.190308087 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls") pod "image-registry-767d584684-w8wk7" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2") : secret "image-registry-tls" not found
Jul 06 17:41:19.597155 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:19.596877    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 17:41:19.597155 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:19.596955    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert podName:f3682cbe-9ac5-4915-9e80-8c897e4a3996 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:35.596939872 +0000 UTC m=+71.190394380 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-7lvhg" (UID: "f3682cbe-9ac5-4915-9e80-8c897e4a3996") : secret "networking-console-plugin-cert" not found
Jul 06 17:41:19.697526 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:19.697499    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:19.697657 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:19.697625    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 17:41:19.697712 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:19.697673    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls podName:a3fe6180-c3f7-4b6f-bb58-591305c33c45 nodeName:}" failed. No retries permitted until 2026-07-06 17:41:35.697662585 +0000 UTC m=+71.291117093 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls") pod "dns-default-97mtx" (UID: "a3fe6180-c3f7-4b6f-bb58-591305c33c45") : secret "dns-default-metrics-tls" not found
Jul 06 17:41:29.061114 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:29.061081    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xqg68"
Jul 06 17:41:30.575142 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:30.575113    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:41:30.575522 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:30.575188    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:41:30.578050 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:30.578027    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 17:41:30.578099 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:30.578059    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 17:41:30.586325 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:30.586308    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 06 17:41:30.586386 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:30.586375    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs podName:311b2011-44ab-4733-a214-04809f2186f6 nodeName:}" failed. No retries permitted until 2026-07-06 17:42:34.586358904 +0000 UTC m=+130.179813413 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs") pod "network-metrics-daemon-fsm9w" (UID: "311b2011-44ab-4733-a214-04809f2186f6") : secret "metrics-daemon-secret" not found
Jul 06 17:41:30.589974 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:30.589955    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d44eac64-207b-4b14-b10b-9d54cd3ae49b-original-pull-secret\") pod \"global-pull-secret-syncer-vpff9\" (UID: \"d44eac64-207b-4b14-b10b-9d54cd3ae49b\") " pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:41:30.621698 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:30.621678    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpff9"
Jul 06 17:41:30.676499 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:30.676473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcgtg\" (UniqueName: \"kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg\") pod \"network-check-target-h7n2n\" (UID: \"74033f51-c97e-44e8-81cc-87ed6a2a086e\") " pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:41:30.679288 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:30.679262    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 17:41:30.689719 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:30.689700    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 17:41:30.700442 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:30.700415    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcgtg\" (UniqueName: \"kubernetes.io/projected/74033f51-c97e-44e8-81cc-87ed6a2a086e-kube-api-access-pcgtg\") pod \"network-check-target-h7n2n\" (UID: \"74033f51-c97e-44e8-81cc-87ed6a2a086e\") " pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:41:30.737599 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:30.737577    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vpff9"]
Jul 06 17:41:30.740355 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:41:30.740334    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd44eac64_207b_4b14_b10b_9d54cd3ae49b.slice/crio-50887f1ea582eaaaf2c6d1d0846e84a89d7e212c469c419a0fb912d6553edf75 WatchSource:0}: Error finding container 50887f1ea582eaaaf2c6d1d0846e84a89d7e212c469c419a0fb912d6553edf75: Status 404 returned error can't find the container with id 50887f1ea582eaaaf2c6d1d0846e84a89d7e212c469c419a0fb912d6553edf75
Jul 06 17:41:30.918165 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:30.918107    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vsjgr\""
Jul 06 17:41:30.926257 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:30.926228    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:41:31.038218 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:31.038163    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h7n2n"]
Jul 06 17:41:31.039942 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:41:31.039919    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74033f51_c97e_44e8_81cc_87ed6a2a086e.slice/crio-080d2bd8ebe98e717475df77ae4c86df7245776efec845d8dca2353b1302fe62 WatchSource:0}: Error finding container 080d2bd8ebe98e717475df77ae4c86df7245776efec845d8dca2353b1302fe62: Status 404 returned error can't find the container with id 080d2bd8ebe98e717475df77ae4c86df7245776efec845d8dca2353b1302fe62
Jul 06 17:41:31.180740 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:31.180663    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h7n2n" event={"ID":"74033f51-c97e-44e8-81cc-87ed6a2a086e","Type":"ContainerStarted","Data":"080d2bd8ebe98e717475df77ae4c86df7245776efec845d8dca2353b1302fe62"}
Jul 06 17:41:31.182162 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:31.182136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vpff9" event={"ID":"d44eac64-207b-4b14-b10b-9d54cd3ae49b","Type":"ContainerStarted","Data":"50887f1ea582eaaaf2c6d1d0846e84a89d7e212c469c419a0fb912d6553edf75"}
Jul 06 17:41:35.615021 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:35.614984    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:41:35.615465 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:35.615068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:41:35.615465 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:35.615174    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 17:41:35.615465 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:35.615197    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-767d584684-w8wk7: secret "image-registry-tls" not found
Jul 06 17:41:35.615465 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:35.615214    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 17:41:35.615465 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:35.615268    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls podName:d99db248-8fac-4e30-acd8-661343bb2ff2 nodeName:}" failed. No retries permitted until 2026-07-06 17:42:07.615247955 +0000 UTC m=+103.208702479 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls") pod "image-registry-767d584684-w8wk7" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2") : secret "image-registry-tls" not found
Jul 06 17:41:35.615465 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:35.615288    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert podName:f3682cbe-9ac5-4915-9e80-8c897e4a3996 nodeName:}" failed. No retries permitted until 2026-07-06 17:42:07.615276805 +0000 UTC m=+103.208731332 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-7lvhg" (UID: "f3682cbe-9ac5-4915-9e80-8c897e4a3996") : secret "networking-console-plugin-cert" not found
Jul 06 17:41:35.716292 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:35.716253    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:41:35.716463 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:35.716409    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 17:41:35.716516 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:41:35.716482    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls podName:a3fe6180-c3f7-4b6f-bb58-591305c33c45 nodeName:}" failed. No retries permitted until 2026-07-06 17:42:07.716461134 +0000 UTC m=+103.309915644 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls") pod "dns-default-97mtx" (UID: "a3fe6180-c3f7-4b6f-bb58-591305c33c45") : secret "dns-default-metrics-tls" not found
Jul 06 17:41:36.205360 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:36.205325    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h7n2n" event={"ID":"74033f51-c97e-44e8-81cc-87ed6a2a086e","Type":"ContainerStarted","Data":"72c8ca24d63f0adc5cce4648f6eb1df8e438354650487cfa96f17e686c27a28f"}
Jul 06 17:41:36.205524 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:36.205446    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:41:36.206875 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:36.206853    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vpff9" event={"ID":"d44eac64-207b-4b14-b10b-9d54cd3ae49b","Type":"ContainerStarted","Data":"bb53fdfed9fdf98aaf42f7554d3bd14e94358f228a946dc591b8a6f87bb7cf37"}
Jul 06 17:41:36.239112 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:36.239071    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vpff9" podStartSLOduration=66.13030261 podStartE2EDuration="1m11.239058534s" totalImagesPullingTime="5.108755924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:36.238543569 +0000 UTC m=+71.831998111" watchObservedRunningTime="2026-07-06 17:41:36.239058534 +0000 UTC m=+71.832513064"
Jul 06 17:41:36.239492 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:41:36.239464    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-h7n2n" podStartSLOduration=66.432460954 podStartE2EDuration="1m11.239454806s" totalImagesPullingTime="4.806993852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:41:36.222779723 +0000 UTC m=+71.816234255" watchObservedRunningTime="2026-07-06 17:41:36.239454806 +0000 UTC m=+71.832909335"
Jul 06 17:42:07.212532 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:42:07.212395    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-h7n2n"
Jul 06 17:42:07.637399 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:42:07.637371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:42:07.637525 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:42:07.637424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:42:07.637525 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:42:07.637519    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 17:42:07.637594 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:42:07.637567    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert podName:f3682cbe-9ac5-4915-9e80-8c897e4a3996 nodeName:}" failed. No retries permitted until 2026-07-06 17:43:11.637552884 +0000 UTC m=+167.231007392 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-7lvhg" (UID: "f3682cbe-9ac5-4915-9e80-8c897e4a3996") : secret "networking-console-plugin-cert" not found
Jul 06 17:42:07.637677 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:42:07.637517    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 17:42:07.637677 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:42:07.637611    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-767d584684-w8wk7: secret "image-registry-tls" not found
Jul 06 17:42:07.637677 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:42:07.637661    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls podName:d99db248-8fac-4e30-acd8-661343bb2ff2 nodeName:}" failed. No retries permitted until 2026-07-06 17:43:11.637646948 +0000 UTC m=+167.231101456 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls") pod "image-registry-767d584684-w8wk7" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2") : secret "image-registry-tls" not found
Jul 06 17:42:07.738147 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:42:07.738078    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:42:07.738259 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:42:07.738209    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 17:42:07.738259 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:42:07.738257    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls podName:a3fe6180-c3f7-4b6f-bb58-591305c33c45 nodeName:}" failed. No retries permitted until 2026-07-06 17:43:11.738245062 +0000 UTC m=+167.331699569 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls") pod "dns-default-97mtx" (UID: "a3fe6180-c3f7-4b6f-bb58-591305c33c45") : secret "dns-default-metrics-tls" not found
Jul 06 17:42:34.619503 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:42:34.619468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:42:34.620000 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:42:34.619585    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 06 17:42:34.620000 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:42:34.619649    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs podName:311b2011-44ab-4733-a214-04809f2186f6 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:36.619633396 +0000 UTC m=+252.213087904 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs") pod "network-metrics-daemon-fsm9w" (UID: "311b2011-44ab-4733-a214-04809f2186f6") : secret "metrics-daemon-secret" not found
Jul 06 17:43:06.774052 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:43:06.774011    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg" podUID="f3682cbe-9ac5-4915-9e80-8c897e4a3996"
Jul 06 17:43:06.781172 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:43:06.781149    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-767d584684-w8wk7" podUID="d99db248-8fac-4e30-acd8-661343bb2ff2"
Jul 06 17:43:06.854938 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:43:06.854889    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-97mtx" podUID="a3fe6180-c3f7-4b6f-bb58-591305c33c45"
Jul 06 17:43:06.925659 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:43:06.925628    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:43:07.483124 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:07.483103    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:43:07.483213 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:07.483103    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:43:07.483298 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:07.483103    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-97mtx"
Jul 06 17:43:11.671302 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:11.671262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:43:11.671699 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:11.671323    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:43:11.671699 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:43:11.671426    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 17:43:11.671699 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:43:11.671426    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 17:43:11.671699 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:43:11.671449    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-767d584684-w8wk7: secret "image-registry-tls" not found
Jul 06 17:43:11.671699 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:43:11.671505    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert podName:f3682cbe-9ac5-4915-9e80-8c897e4a3996 nodeName:}" failed. No retries permitted until 2026-07-06 17:45:13.671488951 +0000 UTC m=+289.264943458 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-7lvhg" (UID: "f3682cbe-9ac5-4915-9e80-8c897e4a3996") : secret "networking-console-plugin-cert" not found
Jul 06 17:43:11.671699 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:43:11.671520    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls podName:d99db248-8fac-4e30-acd8-661343bb2ff2 nodeName:}" failed. No retries permitted until 2026-07-06 17:45:13.671513193 +0000 UTC m=+289.264967701 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls") pod "image-registry-767d584684-w8wk7" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2") : secret "image-registry-tls" not found
Jul 06 17:43:11.772507 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:11.772478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:43:11.772635 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:43:11.772616    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 17:43:11.772700 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:43:11.772688    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls podName:a3fe6180-c3f7-4b6f-bb58-591305c33c45 nodeName:}" failed. No retries permitted until 2026-07-06 17:45:13.772668974 +0000 UTC m=+289.366123483 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls") pod "dns-default-97mtx" (UID: "a3fe6180-c3f7-4b6f-bb58-591305c33c45") : secret "dns-default-metrics-tls" not found
Jul 06 17:43:12.504093 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:12.504060    2581 generic.go:356] "Generic (PLEG): container finished" podID="50fa955c-ef41-4b07-ae5a-04c8919d4dd7" containerID="34b0f5b3e5bacfe14842c5fe0ea2adefcba4f3476c70731e463a29e872fedeab" exitCode=255
Jul 06 17:43:12.504280 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:12.504144    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc" event={"ID":"50fa955c-ef41-4b07-ae5a-04c8919d4dd7","Type":"ContainerDied","Data":"34b0f5b3e5bacfe14842c5fe0ea2adefcba4f3476c70731e463a29e872fedeab"}
Jul 06 17:43:12.504539 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:12.504521    2581 scope.go:117] "RemoveContainer" containerID="34b0f5b3e5bacfe14842c5fe0ea2adefcba4f3476c70731e463a29e872fedeab"
Jul 06 17:43:12.506008 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:12.505942    2581 generic.go:356] "Generic (PLEG): container finished" podID="b6542aba-0caa-4b27-873b-7a46ae3df205" containerID="0424a9b974c126c52a9555d3f01be60cff6b7ccf1fb47c26946548a12353b2d9" exitCode=1
Jul 06 17:43:12.506008 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:12.505982    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp" event={"ID":"b6542aba-0caa-4b27-873b-7a46ae3df205","Type":"ContainerDied","Data":"0424a9b974c126c52a9555d3f01be60cff6b7ccf1fb47c26946548a12353b2d9"}
Jul 06 17:43:12.506323 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:12.506307    2581 scope.go:117] "RemoveContainer" containerID="0424a9b974c126c52a9555d3f01be60cff6b7ccf1fb47c26946548a12353b2d9"
Jul 06 17:43:13.125592 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:13.125561    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:43:13.511735 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:13.511695    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc" event={"ID":"50fa955c-ef41-4b07-ae5a-04c8919d4dd7","Type":"ContainerStarted","Data":"56bdb45e26cb8e01ba34baa3b66d144a88bda0e4aa8d93e2ae9c98c0d7ec871e"}
Jul 06 17:43:13.513643 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:13.513617    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp" event={"ID":"b6542aba-0caa-4b27-873b-7a46ae3df205","Type":"ContainerStarted","Data":"142bca982716b359bd15163742857e783fb2a68d06999ee7858171711ced105e"}
Jul 06 17:43:13.513819 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:13.513806    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:43:13.514411 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:13.514384    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:43:21.906982 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:43:21.906908    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:44:33.511470 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.511433    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7nvq4"]
Jul 06 17:44:33.515276 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.515253    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:33.518032 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.518000    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 17:44:33.518163 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.518128    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 17:44:33.518163 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.518137    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 17:44:33.518305 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.518205    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 17:44:33.519291 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.519270    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xts8l\""
Jul 06 17:44:33.535288 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.535266    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7nvq4"]
Jul 06 17:44:33.622458 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.622432    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f78fad60-12aa-4b66-abce-3d490c40a6c5-data-volume\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:33.622545 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.622466    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:33.622545 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.622535    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f78fad60-12aa-4b66-abce-3d490c40a6c5-crio-socket\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:33.622618 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.622570    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:33.622618 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.622591    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxrhs\" (UniqueName: \"kubernetes.io/projected/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-api-access-mxrhs\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:33.722826 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.722800    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:33.722934 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.722830    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxrhs\" (UniqueName: \"kubernetes.io/projected/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-api-access-mxrhs\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:33.722934 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.722867    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f78fad60-12aa-4b66-abce-3d490c40a6c5-data-volume\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:33.722934 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.722920    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:33.723071 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:33.722935    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 17:44:33.723071 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.722991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f78fad60-12aa-4b66-abce-3d490c40a6c5-crio-socket\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:33.723071 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:33.723008    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm podName:f78fad60-12aa-4b66-abce-3d490c40a6c5 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:34.222988793 +0000 UTC m=+249.816443309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7nvq4" (UID: "f78fad60-12aa-4b66-abce-3d490c40a6c5") : configmap "kube-rbac-proxy" not found
Jul 06 17:44:33.723071 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.723051    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f78fad60-12aa-4b66-abce-3d490c40a6c5-crio-socket\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:33.723274 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:33.723104    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 17:44:33.723274 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:33.723180    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls podName:f78fad60-12aa-4b66-abce-3d490c40a6c5 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:34.223161048 +0000 UTC m=+249.816615573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7nvq4" (UID: "f78fad60-12aa-4b66-abce-3d490c40a6c5") : secret "insights-runtime-extractor-tls" not found
Jul 06 17:44:33.723274 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.723186    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f78fad60-12aa-4b66-abce-3d490c40a6c5-data-volume\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:33.731086 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:33.731067    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxrhs\" (UniqueName: \"kubernetes.io/projected/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-api-access-mxrhs\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:34.228457 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:34.228426    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:34.228615 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:34.228477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:34.228615 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:34.228545    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 17:44:34.228615 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:34.228567    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 17:44:34.228615 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:34.228602    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm podName:f78fad60-12aa-4b66-abce-3d490c40a6c5 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:35.228586346 +0000 UTC m=+250.822040858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7nvq4" (UID: "f78fad60-12aa-4b66-abce-3d490c40a6c5") : configmap "kube-rbac-proxy" not found
Jul 06 17:44:34.228785 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:34.228660    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls podName:f78fad60-12aa-4b66-abce-3d490c40a6c5 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:35.228652936 +0000 UTC m=+250.822107444 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7nvq4" (UID: "f78fad60-12aa-4b66-abce-3d490c40a6c5") : secret "insights-runtime-extractor-tls" not found
Jul 06 17:44:35.235034 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:35.235001    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:35.235421 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:35.235058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:35.235421 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:35.235132    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 17:44:35.235421 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:35.235188    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 17:44:35.235421 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:35.235200    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm podName:f78fad60-12aa-4b66-abce-3d490c40a6c5 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:37.235183895 +0000 UTC m=+252.828638407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7nvq4" (UID: "f78fad60-12aa-4b66-abce-3d490c40a6c5") : configmap "kube-rbac-proxy" not found
Jul 06 17:44:35.235421 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:35.235231    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls podName:f78fad60-12aa-4b66-abce-3d490c40a6c5 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:37.235219437 +0000 UTC m=+252.828673944 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7nvq4" (UID: "f78fad60-12aa-4b66-abce-3d490c40a6c5") : secret "insights-runtime-extractor-tls" not found
Jul 06 17:44:35.551796 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:35.551728    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v58n8_a79ce496-9313-4d12-912a-b17d2429b3a6/dns-node-resolver/0.log"
Jul 06 17:44:36.351587 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:36.351563    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5gcxt_ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd/node-ca/0.log"
Jul 06 17:44:36.644722 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:36.644651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:44:36.644852 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:36.644803    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 06 17:44:36.644916 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:36.644884    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs podName:311b2011-44ab-4733-a214-04809f2186f6 nodeName:}" failed. No retries permitted until 2026-07-06 17:46:38.644868292 +0000 UTC m=+374.238322799 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs") pod "network-metrics-daemon-fsm9w" (UID: "311b2011-44ab-4733-a214-04809f2186f6") : secret "metrics-daemon-secret" not found
Jul 06 17:44:37.249653 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:37.249616    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:37.249815 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:37.249697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:37.249815 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:37.249779    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 17:44:37.249815 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:37.249795    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 17:44:37.250071 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:37.249834    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm podName:f78fad60-12aa-4b66-abce-3d490c40a6c5 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:41.249818577 +0000 UTC m=+256.843273086 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7nvq4" (UID: "f78fad60-12aa-4b66-abce-3d490c40a6c5") : configmap "kube-rbac-proxy" not found
Jul 06 17:44:37.250071 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:37.249870    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls podName:f78fad60-12aa-4b66-abce-3d490c40a6c5 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:41.249849769 +0000 UTC m=+256.843304353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7nvq4" (UID: "f78fad60-12aa-4b66-abce-3d490c40a6c5") : secret "insights-runtime-extractor-tls" not found
Jul 06 17:44:37.551528 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:37.551454    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xnt6h_df45d8cf-5c5e-4e51-bc0a-029de6595c53/serve-healthcheck-canary/0.log"
Jul 06 17:44:41.281136 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:41.281091    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:41.281676 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:41.281234    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 17:44:41.281676 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:41.281262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:41.281676 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:41.281296    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls podName:f78fad60-12aa-4b66-abce-3d490c40a6c5 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:49.281278887 +0000 UTC m=+264.874733399 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7nvq4" (UID: "f78fad60-12aa-4b66-abce-3d490c40a6c5") : secret "insights-runtime-extractor-tls" not found
Jul 06 17:44:41.281676 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:41.281315    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 17:44:41.281676 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:41.281376    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm podName:f78fad60-12aa-4b66-abce-3d490c40a6c5 nodeName:}" failed. No retries permitted until 2026-07-06 17:44:49.281363305 +0000 UTC m=+264.874817816 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7nvq4" (UID: "f78fad60-12aa-4b66-abce-3d490c40a6c5") : configmap "kube-rbac-proxy" not found
Jul 06 17:44:49.332278 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:49.332187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:49.332278 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:49.332251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:44:49.332735 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:49.332379    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 17:44:49.332735 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:44:49.332458    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls podName:f78fad60-12aa-4b66-abce-3d490c40a6c5 nodeName:}" failed. No retries permitted until 2026-07-06 17:45:05.332436921 +0000 UTC m=+280.925891457 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7nvq4" (UID: "f78fad60-12aa-4b66-abce-3d490c40a6c5") : secret "insights-runtime-extractor-tls" not found
Jul 06 17:44:49.332870 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:44:49.332838    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78fad60-12aa-4b66-abce-3d490c40a6c5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:45:05.337780 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:05.337743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:45:05.341620 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:05.341584    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78fad60-12aa-4b66-abce-3d490c40a6c5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7nvq4\" (UID: \"f78fad60-12aa-4b66-abce-3d490c40a6c5\") " pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:45:05.624847 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:05.624786    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7nvq4"
Jul 06 17:45:05.742221 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:05.742194    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7nvq4"]
Jul 06 17:45:05.745063 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:45:05.745037    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf78fad60_12aa_4b66_abce_3d490c40a6c5.slice/crio-16e5a8fc56fd08cf32c86a484465f6886ce7ba8637696736413ee7a6df9d4a78 WatchSource:0}: Error finding container 16e5a8fc56fd08cf32c86a484465f6886ce7ba8637696736413ee7a6df9d4a78: Status 404 returned error can't find the container with id 16e5a8fc56fd08cf32c86a484465f6886ce7ba8637696736413ee7a6df9d4a78
Jul 06 17:45:05.853508 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:05.853474    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7nvq4" event={"ID":"f78fad60-12aa-4b66-abce-3d490c40a6c5","Type":"ContainerStarted","Data":"16e5a8fc56fd08cf32c86a484465f6886ce7ba8637696736413ee7a6df9d4a78"}
Jul 06 17:45:09.869849 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:09.869813    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7nvq4" event={"ID":"f78fad60-12aa-4b66-abce-3d490c40a6c5","Type":"ContainerStarted","Data":"95315db534c6b6588c3792cf5edb3a25b563bfda61d18c0a2948f363e5a74782"}
Jul 06 17:45:10.484373 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:45:10.484335    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-97mtx" podUID="a3fe6180-c3f7-4b6f-bb58-591305c33c45"
Jul 06 17:45:10.484373 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:45:10.484335    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-767d584684-w8wk7" podUID="d99db248-8fac-4e30-acd8-661343bb2ff2"
Jul 06 17:45:10.484559 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:45:10.484335    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg" podUID="f3682cbe-9ac5-4915-9e80-8c897e4a3996"
Jul 06 17:45:10.875148 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:10.875069    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:45:10.875148 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:10.875101    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7nvq4" event={"ID":"f78fad60-12aa-4b66-abce-3d490c40a6c5","Type":"ContainerStarted","Data":"b71f0090a17cf02324da39a6d431cd46cadce26eb5082eb1dd90adfee5e1de62"}
Jul 06 17:45:10.875575 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:10.875223    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-97mtx"
Jul 06 17:45:10.875575 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:10.875410    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:45:12.882880 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:12.882846    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7nvq4" event={"ID":"f78fad60-12aa-4b66-abce-3d490c40a6c5","Type":"ContainerStarted","Data":"90644defdb3e4bcedbec71ef99c37adf84e77f5eb6f102e72e4cb56ed5a10eb9"}
Jul 06 17:45:12.887393 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:12.887364    2581 generic.go:356] "Generic (PLEG): container finished" podID="b6542aba-0caa-4b27-873b-7a46ae3df205" containerID="142bca982716b359bd15163742857e783fb2a68d06999ee7858171711ced105e" exitCode=1
Jul 06 17:45:12.887520 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:12.887443    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp" event={"ID":"b6542aba-0caa-4b27-873b-7a46ae3df205","Type":"ContainerDied","Data":"142bca982716b359bd15163742857e783fb2a68d06999ee7858171711ced105e"}
Jul 06 17:45:12.887520 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:12.887511    2581 scope.go:117] "RemoveContainer" containerID="0424a9b974c126c52a9555d3f01be60cff6b7ccf1fb47c26946548a12353b2d9"
Jul 06 17:45:12.887808 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:12.887786    2581 scope.go:117] "RemoveContainer" containerID="142bca982716b359bd15163742857e783fb2a68d06999ee7858171711ced105e"
Jul 06 17:45:12.888038 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:45:12.888010    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-79b9d59d7b-7c2kp_open-cluster-management-agent-addon(b6542aba-0caa-4b27-873b-7a46ae3df205)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp" podUID="b6542aba-0caa-4b27-873b-7a46ae3df205"
Jul 06 17:45:12.889781 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:12.889759    2581 generic.go:356] "Generic (PLEG): container finished" podID="50fa955c-ef41-4b07-ae5a-04c8919d4dd7" containerID="56bdb45e26cb8e01ba34baa3b66d144a88bda0e4aa8d93e2ae9c98c0d7ec871e" exitCode=255
Jul 06 17:45:12.889886 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:12.889793    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc" event={"ID":"50fa955c-ef41-4b07-ae5a-04c8919d4dd7","Type":"ContainerDied","Data":"56bdb45e26cb8e01ba34baa3b66d144a88bda0e4aa8d93e2ae9c98c0d7ec871e"}
Jul 06 17:45:12.890085 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:12.890071    2581 scope.go:117] "RemoveContainer" containerID="56bdb45e26cb8e01ba34baa3b66d144a88bda0e4aa8d93e2ae9c98c0d7ec871e"
Jul 06 17:45:12.890208 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:45:12.890192    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-7f568d4746-lh6zc_open-cluster-management-agent-addon(50fa955c-ef41-4b07-ae5a-04c8919d4dd7)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc" podUID="50fa955c-ef41-4b07-ae5a-04c8919d4dd7"
Jul 06 17:45:12.896564 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:12.896546    2581 scope.go:117] "RemoveContainer" containerID="34b0f5b3e5bacfe14842c5fe0ea2adefcba4f3476c70731e463a29e872fedeab"
Jul 06 17:45:12.903312 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:12.903262    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7nvq4" podStartSLOduration=33.374878963 podStartE2EDuration="39.903243625s" totalImagesPullingTime="6.528364662s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:44:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:12.9020194 +0000 UTC m=+288.495473932" watchObservedRunningTime="2026-07-06 17:45:12.903243625 +0000 UTC m=+288.496698158"
Jul 06 17:45:13.514147 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:13.514114    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:45:13.697456 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:13.697427    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:45:13.697565 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:13.697485    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:45:13.699948 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:13.699917    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls\") pod \"image-registry-767d584684-w8wk7\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") " pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:45:13.700143 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:13.700120    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f3682cbe-9ac5-4915-9e80-8c897e4a3996-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-7lvhg\" (UID: \"f3682cbe-9ac5-4915-9e80-8c897e4a3996\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:45:13.798248 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:13.798187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:45:13.800219 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:13.800189    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3fe6180-c3f7-4b6f-bb58-591305c33c45-metrics-tls\") pod \"dns-default-97mtx\" (UID: \"a3fe6180-c3f7-4b6f-bb58-591305c33c45\") " pod="openshift-dns/dns-default-97mtx"
Jul 06 17:45:13.880066 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:13.880040    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xsgc2\""
Jul 06 17:45:13.880066 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:13.880046    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v256q\""
Jul 06 17:45:13.880209 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:13.880118    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lmt7g\""
Jul 06 17:45:13.886953 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:13.886924    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-97mtx"
Jul 06 17:45:13.887231 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:13.886958    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:45:13.887231 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:13.886931    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"
Jul 06 17:45:13.897160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:13.897137    2581 scope.go:117] "RemoveContainer" containerID="142bca982716b359bd15163742857e783fb2a68d06999ee7858171711ced105e"
Jul 06 17:45:13.897347 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:45:13.897324    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-79b9d59d7b-7c2kp_open-cluster-management-agent-addon(b6542aba-0caa-4b27-873b-7a46ae3df205)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp" podUID="b6542aba-0caa-4b27-873b-7a46ae3df205"
Jul 06 17:45:14.034634 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:14.034611    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-97mtx"]
Jul 06 17:45:14.037164 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:45:14.037129    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3fe6180_c3f7_4b6f_bb58_591305c33c45.slice/crio-e60036ecf03e2ab39e9cc579b49441b1ffee1e65bdce6d5fda1c0385c5681a3a WatchSource:0}: Error finding container e60036ecf03e2ab39e9cc579b49441b1ffee1e65bdce6d5fda1c0385c5681a3a: Status 404 returned error can't find the container with id e60036ecf03e2ab39e9cc579b49441b1ffee1e65bdce6d5fda1c0385c5681a3a
Jul 06 17:45:14.103745 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:14.103684    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc"
Jul 06 17:45:14.104034 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:14.104017    2581 scope.go:117] "RemoveContainer" containerID="56bdb45e26cb8e01ba34baa3b66d144a88bda0e4aa8d93e2ae9c98c0d7ec871e"
Jul 06 17:45:14.104166 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:45:14.104146    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-7f568d4746-lh6zc_open-cluster-management-agent-addon(50fa955c-ef41-4b07-ae5a-04c8919d4dd7)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc" podUID="50fa955c-ef41-4b07-ae5a-04c8919d4dd7"
Jul 06 17:45:14.139207 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:14.139186    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:45:14.256497 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:14.256472    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg"]
Jul 06 17:45:14.258783 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:45:14.258761    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3682cbe_9ac5_4915_9e80_8c897e4a3996.slice/crio-af31bccf9153d98dd04caee9ec2df90680b63d0feb331495cc0ab6a358d0aaf7 WatchSource:0}: Error finding container af31bccf9153d98dd04caee9ec2df90680b63d0feb331495cc0ab6a358d0aaf7: Status 404 returned error can't find the container with id af31bccf9153d98dd04caee9ec2df90680b63d0feb331495cc0ab6a358d0aaf7
Jul 06 17:45:14.266341 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:14.266298    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-767d584684-w8wk7"]
Jul 06 17:45:14.268424 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:45:14.268406    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd99db248_8fac_4e30_acd8_661343bb2ff2.slice/crio-a3d93ba6124acdcd1475034550d9c6f63e74b27c56d4483a2b639b0e937d1b7b WatchSource:0}: Error finding container a3d93ba6124acdcd1475034550d9c6f63e74b27c56d4483a2b639b0e937d1b7b: Status 404 returned error can't find the container with id a3d93ba6124acdcd1475034550d9c6f63e74b27c56d4483a2b639b0e937d1b7b
Jul 06 17:45:14.901721 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:14.901689    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg" event={"ID":"f3682cbe-9ac5-4915-9e80-8c897e4a3996","Type":"ContainerStarted","Data":"af31bccf9153d98dd04caee9ec2df90680b63d0feb331495cc0ab6a358d0aaf7"}
Jul 06 17:45:14.903351 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:14.903318    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-767d584684-w8wk7" event={"ID":"d99db248-8fac-4e30-acd8-661343bb2ff2","Type":"ContainerStarted","Data":"d85ae4f3cbf03a2849604863059f1c07e03f615553312a304bdf8cf06926ddd9"}
Jul 06 17:45:14.903475 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:14.903369    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-767d584684-w8wk7" event={"ID":"d99db248-8fac-4e30-acd8-661343bb2ff2","Type":"ContainerStarted","Data":"a3d93ba6124acdcd1475034550d9c6f63e74b27c56d4483a2b639b0e937d1b7b"}
Jul 06 17:45:14.903475 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:14.903430    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:45:14.904729 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:14.904698    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-97mtx" event={"ID":"a3fe6180-c3f7-4b6f-bb58-591305c33c45","Type":"ContainerStarted","Data":"e60036ecf03e2ab39e9cc579b49441b1ffee1e65bdce6d5fda1c0385c5681a3a"}
Jul 06 17:45:14.905022 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:14.905006    2581 scope.go:117] "RemoveContainer" containerID="142bca982716b359bd15163742857e783fb2a68d06999ee7858171711ced105e"
Jul 06 17:45:14.905184 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:45:14.905165    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-79b9d59d7b-7c2kp_open-cluster-management-agent-addon(b6542aba-0caa-4b27-873b-7a46ae3df205)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp" podUID="b6542aba-0caa-4b27-873b-7a46ae3df205"
Jul 06 17:45:14.924941 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:14.924877    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-767d584684-w8wk7" podStartSLOduration=289.92485961 podStartE2EDuration="4m49.92485961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:14.922860764 +0000 UTC m=+290.516315295" watchObservedRunningTime="2026-07-06 17:45:14.92485961 +0000 UTC m=+290.518314137"
Jul 06 17:45:15.909142 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:15.909108    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg" event={"ID":"f3682cbe-9ac5-4915-9e80-8c897e4a3996","Type":"ContainerStarted","Data":"6fc65f62ae7c5f92bf5b669ec40c504a85d077d802c434546e11f60fbaf52592"}
Jul 06 17:45:15.925452 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:15.925406    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-7lvhg" podStartSLOduration=285.760879105 podStartE2EDuration="4m46.925389865s" totalImagesPullingTime="1.16451076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:15.923597881 +0000 UTC m=+291.517052412" watchObservedRunningTime="2026-07-06 17:45:15.925389865 +0000 UTC m=+291.518844395"
Jul 06 17:45:16.913474 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:16.913401    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-97mtx" event={"ID":"a3fe6180-c3f7-4b6f-bb58-591305c33c45","Type":"ContainerStarted","Data":"967d25fe82a35eebf0c8509228067ff40153952e27a846fef0cc098684187e3b"}
Jul 06 17:45:16.913474 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:16.913453    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-97mtx" event={"ID":"a3fe6180-c3f7-4b6f-bb58-591305c33c45","Type":"ContainerStarted","Data":"eb2f7f6a0e68780dd9de5989003121ebb583834bdc095fa571dd37cd32100a47"}
Jul 06 17:45:16.930563 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:16.930519    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-97mtx" podStartSLOduration=251.361457464 podStartE2EDuration="4m13.930502927s" totalImagesPullingTime="2.569045463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:16.929003215 +0000 UTC m=+292.522457763" watchObservedRunningTime="2026-07-06 17:45:16.930502927 +0000 UTC m=+292.523957460"
Jul 06 17:45:17.916959 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:17.916925    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-97mtx"
Jul 06 17:45:22.993175 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:22.993145    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kv695"]
Jul 06 17:45:22.996038 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:22.996017    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.001031 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.001008    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 17:45:23.001170 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.001064    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jpmth\""
Jul 06 17:45:23.001866 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.001848    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 17:45:23.002119 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.002100    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 17:45:23.002198 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.002144    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 17:45:23.002198 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.002166    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 17:45:23.002326 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.002193    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 17:45:23.063540 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.063517    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-textfile\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.063645 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.063541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/905c962d-d69e-4b0a-ae61-27c38fdcd15f-metrics-client-ca\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.063645 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.063562    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-tls\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.063645 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.063581    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9jlt\" (UniqueName: \"kubernetes.io/projected/905c962d-d69e-4b0a-ae61-27c38fdcd15f-kube-api-access-j9jlt\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.063813 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.063674    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-wtmp\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.063813 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.063708    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-accelerators-collector-config\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.063813 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.063754    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.063813 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.063801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/905c962d-d69e-4b0a-ae61-27c38fdcd15f-root\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.063997 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.063825    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/905c962d-d69e-4b0a-ae61-27c38fdcd15f-sys\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.164432 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.164407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9jlt\" (UniqueName: \"kubernetes.io/projected/905c962d-d69e-4b0a-ae61-27c38fdcd15f-kube-api-access-j9jlt\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.164537 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.164439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-wtmp\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.164537 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.164458    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-accelerators-collector-config\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.164537 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.164478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.164698 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.164619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/905c962d-d69e-4b0a-ae61-27c38fdcd15f-root\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.164698 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.164633    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-wtmp\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.164698 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.164645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/905c962d-d69e-4b0a-ae61-27c38fdcd15f-sys\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.164698 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.164689    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/905c962d-d69e-4b0a-ae61-27c38fdcd15f-sys\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.164871 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.164696    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/905c962d-d69e-4b0a-ae61-27c38fdcd15f-root\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.164871 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.164711    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-textfile\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.164871 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.164734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/905c962d-d69e-4b0a-ae61-27c38fdcd15f-metrics-client-ca\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.164871 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.164756    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-tls\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.164871 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:45:23.164835    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 06 17:45:23.165115 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:45:23.164890    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-tls podName:905c962d-d69e-4b0a-ae61-27c38fdcd15f nodeName:}" failed. No retries permitted until 2026-07-06 17:45:23.664871918 +0000 UTC m=+299.258326426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-tls") pod "node-exporter-kv695" (UID: "905c962d-d69e-4b0a-ae61-27c38fdcd15f") : secret "node-exporter-tls" not found
Jul 06 17:45:23.165115 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.165089    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-textfile\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.165212 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.165120    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-accelerators-collector-config\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.165295 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.165276    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/905c962d-d69e-4b0a-ae61-27c38fdcd15f-metrics-client-ca\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.166907 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.166882    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.173186 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.173165    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9jlt\" (UniqueName: \"kubernetes.io/projected/905c962d-d69e-4b0a-ae61-27c38fdcd15f-kube-api-access-j9jlt\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.668101 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.668071    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-tls\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.670319 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.670280    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/905c962d-d69e-4b0a-ae61-27c38fdcd15f-node-exporter-tls\") pod \"node-exporter-kv695\" (UID: \"905c962d-d69e-4b0a-ae61-27c38fdcd15f\") " pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.908764 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.908737    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kv695"
Jul 06 17:45:23.920154 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:45:23.920094    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod905c962d_d69e_4b0a_ae61_27c38fdcd15f.slice/crio-6b304d0f7e570ca37c09333900345019786360499f9a3d3a828316e76040c97c WatchSource:0}: Error finding container 6b304d0f7e570ca37c09333900345019786360499f9a3d3a828316e76040c97c: Status 404 returned error can't find the container with id 6b304d0f7e570ca37c09333900345019786360499f9a3d3a828316e76040c97c
Jul 06 17:45:23.936008 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:23.935979    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kv695" event={"ID":"905c962d-d69e-4b0a-ae61-27c38fdcd15f","Type":"ContainerStarted","Data":"6b304d0f7e570ca37c09333900345019786360499f9a3d3a828316e76040c97c"}
Jul 06 17:45:24.068567 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:24.068530    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr" podUID="798f1d53-692a-4d26-be8c-6118ebf9b4c6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 06 17:45:24.779111 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:24.779089    2581 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 17:45:24.907328 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:45:24.907299    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fsm9w" podUID="311b2011-44ab-4733-a214-04809f2186f6"
Jul 06 17:45:25.906951 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:25.906928    2581 scope.go:117] "RemoveContainer" containerID="142bca982716b359bd15163742857e783fb2a68d06999ee7858171711ced105e"
Jul 06 17:45:25.944208 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:25.944186    2581 generic.go:356] "Generic (PLEG): container finished" podID="905c962d-d69e-4b0a-ae61-27c38fdcd15f" containerID="e399d391b8b6c9feba7e83d468c809eb279d3908395383624290f9362a3b6b96" exitCode=0
Jul 06 17:45:25.944310 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:25.944240    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kv695" event={"ID":"905c962d-d69e-4b0a-ae61-27c38fdcd15f","Type":"ContainerDied","Data":"e399d391b8b6c9feba7e83d468c809eb279d3908395383624290f9362a3b6b96"}
Jul 06 17:45:26.948474 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:26.948423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp" event={"ID":"b6542aba-0caa-4b27-873b-7a46ae3df205","Type":"ContainerStarted","Data":"f03838faaf6f9092020a32724ff7967b66b07a00dd5fdf8450f2eb823478eb5e"}
Jul 06 17:45:26.948915 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:26.948759    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:45:26.950292 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:26.950259    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-79b9d59d7b-7c2kp"
Jul 06 17:45:26.950947 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:26.950920    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kv695" event={"ID":"905c962d-d69e-4b0a-ae61-27c38fdcd15f","Type":"ContainerStarted","Data":"fdeeb71dc79e4d1ed89c2d15ba941820fc36135af3e66d941555bb71ea655c38"}
Jul 06 17:45:26.951103 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:26.950974    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kv695" event={"ID":"905c962d-d69e-4b0a-ae61-27c38fdcd15f","Type":"ContainerStarted","Data":"935ce1cc81b703ade50a116b89c327e11d92d8522aa66ebe20e29806e6712542"}
Jul 06 17:45:26.998125 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:26.998088    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kv695" podStartSLOduration=3.560786426 podStartE2EDuration="4.998073844s" totalImagesPullingTime="1.376234532s" totalInitContainerRuntime="61.052886ms" isStatefulPod=true podCreationTimestamp="2026-07-06 17:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:45:26.996938663 +0000 UTC m=+302.590393193" watchObservedRunningTime="2026-07-06 17:45:26.998073844 +0000 UTC m=+302.591528376"
Jul 06 17:45:27.923232 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:27.923202    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-97mtx"
Jul 06 17:45:28.906703 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:28.906671    2581 scope.go:117] "RemoveContainer" containerID="56bdb45e26cb8e01ba34baa3b66d144a88bda0e4aa8d93e2ae9c98c0d7ec871e"
Jul 06 17:45:29.964884 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:29.964849    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f568d4746-lh6zc" event={"ID":"50fa955c-ef41-4b07-ae5a-04c8919d4dd7","Type":"ContainerStarted","Data":"89824c3b48d76c106c6ebbff28ff794203fedfaffdc4afca8620e53b3f09f151"}
Jul 06 17:45:33.891704 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:33.891664    2581 patch_prober.go:28] interesting pod/image-registry-767d584684-w8wk7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 06 17:45:33.892139 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:33.891719    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-767d584684-w8wk7" podUID="d99db248-8fac-4e30-acd8-661343bb2ff2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 17:45:34.068535 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:34.068499    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr" podUID="798f1d53-692a-4d26-be8c-6118ebf9b4c6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 06 17:45:35.906302 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:35.906255    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:45:35.913787 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:35.913755    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:45:44.068915 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:44.068860    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr" podUID="798f1d53-692a-4d26-be8c-6118ebf9b4c6" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 06 17:45:44.069370 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:44.068955    2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr"
Jul 06 17:45:44.069455 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:44.069418    2581 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"5d716c03fb12e949b152ba20ae25cd1f5a63309e8a84bf2d7947180d079bb4bc"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 06 17:45:44.069496 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:44.069478    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr" podUID="798f1d53-692a-4d26-be8c-6118ebf9b4c6" containerName="service-proxy" containerID="cri-o://5d716c03fb12e949b152ba20ae25cd1f5a63309e8a84bf2d7947180d079bb4bc" gracePeriod=30
Jul 06 17:45:45.016856 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:45.016826    2581 generic.go:356] "Generic (PLEG): container finished" podID="798f1d53-692a-4d26-be8c-6118ebf9b4c6" containerID="5d716c03fb12e949b152ba20ae25cd1f5a63309e8a84bf2d7947180d079bb4bc" exitCode=2
Jul 06 17:45:45.017031 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:45.016884    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr" event={"ID":"798f1d53-692a-4d26-be8c-6118ebf9b4c6","Type":"ContainerDied","Data":"5d716c03fb12e949b152ba20ae25cd1f5a63309e8a84bf2d7947180d079bb4bc"}
Jul 06 17:45:45.017031 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:45.016930    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d567cfbc-ckjcr" event={"ID":"798f1d53-692a-4d26-be8c-6118ebf9b4c6","Type":"ContainerStarted","Data":"3c8663d2d56bb4ac99d1e51cbb93f81d6e989fd4be43c5602162f5c7c372920f"}
Jul 06 17:45:45.572220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:45.572189    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-767d584684-w8wk7"]
Jul 06 17:45:48.745742 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:45:48.745703    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xnt6h_df45d8cf-5c5e-4e51-bc0a-029de6595c53/serve-healthcheck-canary/0.log"
Jul 06 17:46:10.592103 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.592064    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-767d584684-w8wk7" podUID="d99db248-8fac-4e30-acd8-661343bb2ff2" containerName="registry" containerID="cri-o://d85ae4f3cbf03a2849604863059f1c07e03f615553312a304bdf8cf06926ddd9" gracePeriod=30
Jul 06 17:46:10.828122 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.828097    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:46:10.981878 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.981854    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d99db248-8fac-4e30-acd8-661343bb2ff2-ca-trust-extracted\") pod \"d99db248-8fac-4e30-acd8-661343bb2ff2\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") "
Jul 06 17:46:10.982030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.981911    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls\") pod \"d99db248-8fac-4e30-acd8-661343bb2ff2\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") "
Jul 06 17:46:10.982030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.981941    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d99db248-8fac-4e30-acd8-661343bb2ff2-installation-pull-secrets\") pod \"d99db248-8fac-4e30-acd8-661343bb2ff2\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") "
Jul 06 17:46:10.982030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.981958    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79rdh\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-kube-api-access-79rdh\") pod \"d99db248-8fac-4e30-acd8-661343bb2ff2\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") "
Jul 06 17:46:10.982030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.981983    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-certificates\") pod \"d99db248-8fac-4e30-acd8-661343bb2ff2\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") "
Jul 06 17:46:10.982030 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.982013    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d99db248-8fac-4e30-acd8-661343bb2ff2-trusted-ca\") pod \"d99db248-8fac-4e30-acd8-661343bb2ff2\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") "
Jul 06 17:46:10.982339 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.982048    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-bound-sa-token\") pod \"d99db248-8fac-4e30-acd8-661343bb2ff2\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") "
Jul 06 17:46:10.982339 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.982076    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d99db248-8fac-4e30-acd8-661343bb2ff2-image-registry-private-configuration\") pod \"d99db248-8fac-4e30-acd8-661343bb2ff2\" (UID: \"d99db248-8fac-4e30-acd8-661343bb2ff2\") "
Jul 06 17:46:10.982488 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.982456    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d99db248-8fac-4e30-acd8-661343bb2ff2" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 17:46:10.982547 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.982463    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d99db248-8fac-4e30-acd8-661343bb2ff2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d99db248-8fac-4e30-acd8-661343bb2ff2" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 17:46:10.984430 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.984398    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d99db248-8fac-4e30-acd8-661343bb2ff2" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:46:10.984569 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.984468    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d99db248-8fac-4e30-acd8-661343bb2ff2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d99db248-8fac-4e30-acd8-661343bb2ff2" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 17:46:10.984737 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.984708    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d99db248-8fac-4e30-acd8-661343bb2ff2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d99db248-8fac-4e30-acd8-661343bb2ff2" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 17:46:10.984847 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.984777    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d99db248-8fac-4e30-acd8-661343bb2ff2" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:46:10.984974 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.984945    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-kube-api-access-79rdh" (OuterVolumeSpecName: "kube-api-access-79rdh") pod "d99db248-8fac-4e30-acd8-661343bb2ff2" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2"). InnerVolumeSpecName "kube-api-access-79rdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:46:10.991711 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:10.991686    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d99db248-8fac-4e30-acd8-661343bb2ff2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d99db248-8fac-4e30-acd8-661343bb2ff2" (UID: "d99db248-8fac-4e30-acd8-661343bb2ff2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 17:46:11.083067 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.083037    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-bound-sa-token\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:46:11.083067 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.083066    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d99db248-8fac-4e30-acd8-661343bb2ff2-image-registry-private-configuration\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:46:11.083067 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.083075    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d99db248-8fac-4e30-acd8-661343bb2ff2-ca-trust-extracted\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:46:11.083220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.083085    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-tls\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:46:11.083220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.083094    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d99db248-8fac-4e30-acd8-661343bb2ff2-installation-pull-secrets\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:46:11.083220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.083103    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79rdh\" (UniqueName: \"kubernetes.io/projected/d99db248-8fac-4e30-acd8-661343bb2ff2-kube-api-access-79rdh\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:46:11.083220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.083111    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d99db248-8fac-4e30-acd8-661343bb2ff2-registry-certificates\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:46:11.083220 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.083119    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d99db248-8fac-4e30-acd8-661343bb2ff2-trusted-ca\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:46:11.104469 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.104438    2581 generic.go:356] "Generic (PLEG): container finished" podID="d99db248-8fac-4e30-acd8-661343bb2ff2" containerID="d85ae4f3cbf03a2849604863059f1c07e03f615553312a304bdf8cf06926ddd9" exitCode=0
Jul 06 17:46:11.104553 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.104491    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-767d584684-w8wk7"
Jul 06 17:46:11.104553 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.104523    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-767d584684-w8wk7" event={"ID":"d99db248-8fac-4e30-acd8-661343bb2ff2","Type":"ContainerDied","Data":"d85ae4f3cbf03a2849604863059f1c07e03f615553312a304bdf8cf06926ddd9"}
Jul 06 17:46:11.104632 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.104572    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-767d584684-w8wk7" event={"ID":"d99db248-8fac-4e30-acd8-661343bb2ff2","Type":"ContainerDied","Data":"a3d93ba6124acdcd1475034550d9c6f63e74b27c56d4483a2b639b0e937d1b7b"}
Jul 06 17:46:11.104632 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.104592    2581 scope.go:117] "RemoveContainer" containerID="d85ae4f3cbf03a2849604863059f1c07e03f615553312a304bdf8cf06926ddd9"
Jul 06 17:46:11.112330 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.112308    2581 scope.go:117] "RemoveContainer" containerID="d85ae4f3cbf03a2849604863059f1c07e03f615553312a304bdf8cf06926ddd9"
Jul 06 17:46:11.112605 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:46:11.112588    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d85ae4f3cbf03a2849604863059f1c07e03f615553312a304bdf8cf06926ddd9\": container with ID starting with d85ae4f3cbf03a2849604863059f1c07e03f615553312a304bdf8cf06926ddd9 not found: ID does not exist" containerID="d85ae4f3cbf03a2849604863059f1c07e03f615553312a304bdf8cf06926ddd9"
Jul 06 17:46:11.112654 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.112613    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d85ae4f3cbf03a2849604863059f1c07e03f615553312a304bdf8cf06926ddd9"} err="failed to get container status \"d85ae4f3cbf03a2849604863059f1c07e03f615553312a304bdf8cf06926ddd9\": rpc error: code = NotFound desc = could not find container \"d85ae4f3cbf03a2849604863059f1c07e03f615553312a304bdf8cf06926ddd9\": container with ID starting with d85ae4f3cbf03a2849604863059f1c07e03f615553312a304bdf8cf06926ddd9 not found: ID does not exist"
Jul 06 17:46:11.125040 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.125020    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-767d584684-w8wk7"]
Jul 06 17:46:11.128149 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:11.128131    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-767d584684-w8wk7"]
Jul 06 17:46:12.910934 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:12.910879    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d99db248-8fac-4e30-acd8-661343bb2ff2" path="/var/lib/kubelet/pods/d99db248-8fac-4e30-acd8-661343bb2ff2/volumes"
Jul 06 17:46:38.661069 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:38.661019    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:46:38.663506 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:38.663479    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/311b2011-44ab-4733-a214-04809f2186f6-metrics-certs\") pod \"network-metrics-daemon-fsm9w\" (UID: \"311b2011-44ab-4733-a214-04809f2186f6\") " pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:46:38.909178 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:38.909147    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gwzng\""
Jul 06 17:46:38.917345 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:38.917289    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fsm9w"
Jul 06 17:46:39.039400 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:39.039341    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fsm9w"]
Jul 06 17:46:39.041640 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:46:39.041615    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod311b2011_44ab_4733_a214_04809f2186f6.slice/crio-3d3a40bfb4c35dac7b7b79cd33bb199059df9ea1704c9a4acba24a198e1380a2 WatchSource:0}: Error finding container 3d3a40bfb4c35dac7b7b79cd33bb199059df9ea1704c9a4acba24a198e1380a2: Status 404 returned error can't find the container with id 3d3a40bfb4c35dac7b7b79cd33bb199059df9ea1704c9a4acba24a198e1380a2
Jul 06 17:46:39.043537 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:39.043516    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 17:46:39.199242 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:39.199211    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fsm9w" event={"ID":"311b2011-44ab-4733-a214-04809f2186f6","Type":"ContainerStarted","Data":"3d3a40bfb4c35dac7b7b79cd33bb199059df9ea1704c9a4acba24a198e1380a2"}
Jul 06 17:46:40.204640 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:40.204607    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fsm9w" event={"ID":"311b2011-44ab-4733-a214-04809f2186f6","Type":"ContainerStarted","Data":"556f6db0d8e0a9a944cf5923ffec5de8b68cf99ceddfd58eebf3c6602a1586ce"}
Jul 06 17:46:41.212137 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:41.212089    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fsm9w" event={"ID":"311b2011-44ab-4733-a214-04809f2186f6","Type":"ContainerStarted","Data":"d72c0f328ca6d3a7b535d58254cc4e53fbc43dab5dcac094fafbe40eead2c47b"}
Jul 06 17:46:41.228649 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:46:41.228590    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fsm9w" podStartSLOduration=375.231352329 podStartE2EDuration="6m16.228572387s" totalImagesPullingTime="997.220058ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:46:41.228166411 +0000 UTC m=+376.821620943" watchObservedRunningTime="2026-07-06 17:46:41.228572387 +0000 UTC m=+376.822026918"
Jul 06 17:48:56.195498 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.195468    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f86nd"]
Jul 06 17:48:56.195913 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.195672    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d99db248-8fac-4e30-acd8-661343bb2ff2" containerName="registry"
Jul 06 17:48:56.195913 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.195683    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d99db248-8fac-4e30-acd8-661343bb2ff2" containerName="registry"
Jul 06 17:48:56.195913 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.195734    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d99db248-8fac-4e30-acd8-661343bb2ff2" containerName="registry"
Jul 06 17:48:56.198224 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.198208    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-f86nd"
Jul 06 17:48:56.200817 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.200789    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 06 17:48:56.200957 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.200833    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-h5lcq\""
Jul 06 17:48:56.200957 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.200871    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 06 17:48:56.213271 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.213249    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f86nd"]
Jul 06 17:48:56.333074 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.333046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7qk2\" (UniqueName: \"kubernetes.io/projected/f35df168-7dc7-4445-b39a-3045e46ed47e-kube-api-access-c7qk2\") pod \"cert-manager-55b4d954d5-f86nd\" (UID: \"f35df168-7dc7-4445-b39a-3045e46ed47e\") " pod="cert-manager/cert-manager-55b4d954d5-f86nd"
Jul 06 17:48:56.333188 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.333084    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f35df168-7dc7-4445-b39a-3045e46ed47e-bound-sa-token\") pod \"cert-manager-55b4d954d5-f86nd\" (UID: \"f35df168-7dc7-4445-b39a-3045e46ed47e\") " pod="cert-manager/cert-manager-55b4d954d5-f86nd"
Jul 06 17:48:56.433508 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.433479    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7qk2\" (UniqueName: \"kubernetes.io/projected/f35df168-7dc7-4445-b39a-3045e46ed47e-kube-api-access-c7qk2\") pod \"cert-manager-55b4d954d5-f86nd\" (UID: \"f35df168-7dc7-4445-b39a-3045e46ed47e\") " pod="cert-manager/cert-manager-55b4d954d5-f86nd"
Jul 06 17:48:56.433609 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.433515    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f35df168-7dc7-4445-b39a-3045e46ed47e-bound-sa-token\") pod \"cert-manager-55b4d954d5-f86nd\" (UID: \"f35df168-7dc7-4445-b39a-3045e46ed47e\") " pod="cert-manager/cert-manager-55b4d954d5-f86nd"
Jul 06 17:48:56.441737 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.441699    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f35df168-7dc7-4445-b39a-3045e46ed47e-bound-sa-token\") pod \"cert-manager-55b4d954d5-f86nd\" (UID: \"f35df168-7dc7-4445-b39a-3045e46ed47e\") " pod="cert-manager/cert-manager-55b4d954d5-f86nd"
Jul 06 17:48:56.441839 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.441820    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7qk2\" (UniqueName: \"kubernetes.io/projected/f35df168-7dc7-4445-b39a-3045e46ed47e-kube-api-access-c7qk2\") pod \"cert-manager-55b4d954d5-f86nd\" (UID: \"f35df168-7dc7-4445-b39a-3045e46ed47e\") " pod="cert-manager/cert-manager-55b4d954d5-f86nd"
Jul 06 17:48:56.514666 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.514612    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-f86nd"
Jul 06 17:48:56.640042 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.640014    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f86nd"]
Jul 06 17:48:56.643457 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:48:56.643431    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf35df168_7dc7_4445_b39a_3045e46ed47e.slice/crio-3490569723e091518fd3659110759e3aa9aa77d0dba4c3ccea5e1e8490b64035 WatchSource:0}: Error finding container 3490569723e091518fd3659110759e3aa9aa77d0dba4c3ccea5e1e8490b64035: Status 404 returned error can't find the container with id 3490569723e091518fd3659110759e3aa9aa77d0dba4c3ccea5e1e8490b64035
Jul 06 17:48:56.677481 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.677457    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-f86nd" event={"ID":"f35df168-7dc7-4445-b39a-3045e46ed47e","Type":"ContainerStarted","Data":"3490569723e091518fd3659110759e3aa9aa77d0dba4c3ccea5e1e8490b64035"}
Jul 06 17:48:56.694851 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.694827    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhzmc"]
Jul 06 17:48:56.698812 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.698798    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhzmc"
Jul 06 17:48:56.701350 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.701330    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 06 17:48:56.701475 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.701453    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 06 17:48:56.702684 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.702661    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-nxlmp\""
Jul 06 17:48:56.706623 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.706605    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhzmc"]
Jul 06 17:48:56.836682 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.836626    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a96b501d-06cd-4cb1-adc7-6c11ea466ca6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nhzmc\" (UID: \"a96b501d-06cd-4cb1-adc7-6c11ea466ca6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhzmc"
Jul 06 17:48:56.836682 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.836658    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4gsz\" (UniqueName: \"kubernetes.io/projected/a96b501d-06cd-4cb1-adc7-6c11ea466ca6-kube-api-access-r4gsz\") pod \"openshift-lws-operator-fd8ccff4c-nhzmc\" (UID: \"a96b501d-06cd-4cb1-adc7-6c11ea466ca6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhzmc"
Jul 06 17:48:56.937588 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.937563    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a96b501d-06cd-4cb1-adc7-6c11ea466ca6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nhzmc\" (UID: \"a96b501d-06cd-4cb1-adc7-6c11ea466ca6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhzmc"
Jul 06 17:48:56.937689 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.937597    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4gsz\" (UniqueName: \"kubernetes.io/projected/a96b501d-06cd-4cb1-adc7-6c11ea466ca6-kube-api-access-r4gsz\") pod \"openshift-lws-operator-fd8ccff4c-nhzmc\" (UID: \"a96b501d-06cd-4cb1-adc7-6c11ea466ca6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhzmc"
Jul 06 17:48:56.937956 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.937936    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a96b501d-06cd-4cb1-adc7-6c11ea466ca6-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nhzmc\" (UID: \"a96b501d-06cd-4cb1-adc7-6c11ea466ca6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhzmc"
Jul 06 17:48:56.945517 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:56.945489    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4gsz\" (UniqueName: \"kubernetes.io/projected/a96b501d-06cd-4cb1-adc7-6c11ea466ca6-kube-api-access-r4gsz\") pod \"openshift-lws-operator-fd8ccff4c-nhzmc\" (UID: \"a96b501d-06cd-4cb1-adc7-6c11ea466ca6\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhzmc"
Jul 06 17:48:57.008122 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:57.008095    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhzmc"
Jul 06 17:48:57.129663 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:57.129623    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhzmc"]
Jul 06 17:48:57.131635 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:48:57.131609    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda96b501d_06cd_4cb1_adc7_6c11ea466ca6.slice/crio-723e0e350ff6e62cc14def93e7e07c26616b37dba4754405750acd28f837cb1b WatchSource:0}: Error finding container 723e0e350ff6e62cc14def93e7e07c26616b37dba4754405750acd28f837cb1b: Status 404 returned error can't find the container with id 723e0e350ff6e62cc14def93e7e07c26616b37dba4754405750acd28f837cb1b
Jul 06 17:48:57.686266 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:57.686215    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhzmc" event={"ID":"a96b501d-06cd-4cb1-adc7-6c11ea466ca6","Type":"ContainerStarted","Data":"723e0e350ff6e62cc14def93e7e07c26616b37dba4754405750acd28f837cb1b"}
Jul 06 17:48:59.696703 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:59.696670    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-f86nd" event={"ID":"f35df168-7dc7-4445-b39a-3045e46ed47e","Type":"ContainerStarted","Data":"70ceb2bf1d4ac72de7a0dabefffcde4591144932c148e53dfa9577d8fb040d5e"}
Jul 06 17:48:59.712406 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:48:59.712362    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-f86nd" podStartSLOduration=0.794002824 podStartE2EDuration="3.712347243s" totalImagesPullingTime="2.918344419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:48:59.711843463 +0000 UTC m=+515.305298006" watchObservedRunningTime="2026-07-06 17:48:59.712347243 +0000 UTC m=+515.305801773"
Jul 06 17:49:12.746670 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:12.746639    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhzmc" event={"ID":"a96b501d-06cd-4cb1-adc7-6c11ea466ca6","Type":"ContainerStarted","Data":"d68f2e15815e33c18afb997ac98e7f04ad4ad911ec49a0d2606d5986a9908a29"}
Jul 06 17:49:12.765583 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:12.765528    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhzmc" podStartSLOduration=1.7388117699999999 podStartE2EDuration="16.7655097s" totalImagesPullingTime="15.02669793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:49:12.76507715 +0000 UTC m=+528.358531681" watchObservedRunningTime="2026-07-06 17:49:12.7655097 +0000 UTC m=+528.358964228"
Jul 06 17:49:28.152090 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.152002    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"]
Jul 06 17:49:28.155454 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.155430    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.158462 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.158444    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mmw7p\""
Jul 06 17:49:28.158543 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.158469    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 17:49:28.158882 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.158861    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 17:49:28.159014 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.158950    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 17:49:28.159014 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.158983    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 06 17:49:28.172480 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.172457    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"]
Jul 06 17:49:28.239716 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.239692    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5389f834-f90e-4388-b2df-fbf5ea3ffe55-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.239805 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.239721    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qlgt\" (UniqueName: \"kubernetes.io/projected/5389f834-f90e-4388-b2df-fbf5ea3ffe55-kube-api-access-5qlgt\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.239805 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.239744    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5389f834-f90e-4388-b2df-fbf5ea3ffe55-webhook-cert\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.239805 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.239773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5389f834-f90e-4388-b2df-fbf5ea3ffe55-tmp\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.239928 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.239830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5389f834-f90e-4388-b2df-fbf5ea3ffe55-manifests\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.340710 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.340689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5389f834-f90e-4388-b2df-fbf5ea3ffe55-tmp\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.340814 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.340720    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5389f834-f90e-4388-b2df-fbf5ea3ffe55-manifests\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.340814 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.340742    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5389f834-f90e-4388-b2df-fbf5ea3ffe55-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.340814 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.340759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qlgt\" (UniqueName: \"kubernetes.io/projected/5389f834-f90e-4388-b2df-fbf5ea3ffe55-kube-api-access-5qlgt\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.340814 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.340787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5389f834-f90e-4388-b2df-fbf5ea3ffe55-webhook-cert\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.341150 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.341127    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5389f834-f90e-4388-b2df-fbf5ea3ffe55-manifests\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.342971 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.342940    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5389f834-f90e-4388-b2df-fbf5ea3ffe55-tmp\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.343272 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.343253    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5389f834-f90e-4388-b2df-fbf5ea3ffe55-webhook-cert\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.343336 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.343255    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5389f834-f90e-4388-b2df-fbf5ea3ffe55-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.349567 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.349543    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qlgt\" (UniqueName: \"kubernetes.io/projected/5389f834-f90e-4388-b2df-fbf5ea3ffe55-kube-api-access-5qlgt\") pod \"opendatahub-operator-controller-manager-74668f54-rpscx\" (UID: \"5389f834-f90e-4388-b2df-fbf5ea3ffe55\") " pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.465565 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.465541    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:28.592728 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.592696    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"]
Jul 06 17:49:28.596856 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:49:28.596827    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5389f834_f90e_4388_b2df_fbf5ea3ffe55.slice/crio-55316723b7d78f00dfa8a4c97e7cb82f2e8af1b600962f189a2cb80ec475963a WatchSource:0}: Error finding container 55316723b7d78f00dfa8a4c97e7cb82f2e8af1b600962f189a2cb80ec475963a: Status 404 returned error can't find the container with id 55316723b7d78f00dfa8a4c97e7cb82f2e8af1b600962f189a2cb80ec475963a
Jul 06 17:49:28.804186 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:28.804118    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx" event={"ID":"5389f834-f90e-4388-b2df-fbf5ea3ffe55","Type":"ContainerStarted","Data":"55316723b7d78f00dfa8a4c97e7cb82f2e8af1b600962f189a2cb80ec475963a"}
Jul 06 17:49:31.818278 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:31.818242    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx" event={"ID":"5389f834-f90e-4388-b2df-fbf5ea3ffe55","Type":"ContainerStarted","Data":"9916b735784339cba23661b0855c5a3da527f7772316207234b5515f667e9f9d"}
Jul 06 17:49:33.827250 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:33.827212    2581 generic.go:356] "Generic (PLEG): container finished" podID="5389f834-f90e-4388-b2df-fbf5ea3ffe55" containerID="9916b735784339cba23661b0855c5a3da527f7772316207234b5515f667e9f9d" exitCode=0
Jul 06 17:49:33.827732 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:33.827269    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx" event={"ID":"5389f834-f90e-4388-b2df-fbf5ea3ffe55","Type":"ContainerDied","Data":"9916b735784339cba23661b0855c5a3da527f7772316207234b5515f667e9f9d"}
Jul 06 17:49:34.833592 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:34.833555    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx" event={"ID":"5389f834-f90e-4388-b2df-fbf5ea3ffe55","Type":"ContainerStarted","Data":"e3124f7248139c9571053462407b4c1bd440d4f5146e8e1e040c31da0814d050"}
Jul 06 17:49:34.834068 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:34.833796    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:34.855805 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:34.855756    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx" podStartSLOduration=2.365534082 podStartE2EDuration="6.85573846s" totalImagesPullingTime="3.439818745s" totalInitContainerRuntime="1.050385633s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:49:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:49:34.853292577 +0000 UTC m=+550.446747105" watchObservedRunningTime="2026-07-06 17:49:34.85573846 +0000 UTC m=+550.449192989"
Jul 06 17:49:45.839510 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:45.839480    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-74668f54-rpscx"
Jul 06 17:49:47.988526 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:47.988490    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-98dc6956-pmkj5"]
Jul 06 17:49:47.996105 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:47.996083    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-98dc6956-pmkj5"
Jul 06 17:49:47.998714 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:47.998696    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 06 17:49:47.998887 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:47.998869    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 06 17:49:47.998966 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:47.998938    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dfr6c\""
Jul 06 17:49:48.000098 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.000083    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 17:49:48.000158 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.000116    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 17:49:48.005011 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.004910    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-98dc6956-pmkj5"]
Jul 06 17:49:48.176182 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.176162    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92359562-7966-400d-87ed-d7d35b885d40-tls-certs\") pod \"kube-auth-proxy-98dc6956-pmkj5\" (UID: \"92359562-7966-400d-87ed-d7d35b885d40\") " pod="openshift-ingress/kube-auth-proxy-98dc6956-pmkj5"
Jul 06 17:49:48.176294 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.176195    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x8jq\" (UniqueName: \"kubernetes.io/projected/92359562-7966-400d-87ed-d7d35b885d40-kube-api-access-7x8jq\") pod \"kube-auth-proxy-98dc6956-pmkj5\" (UID: \"92359562-7966-400d-87ed-d7d35b885d40\") " pod="openshift-ingress/kube-auth-proxy-98dc6956-pmkj5"
Jul 06 17:49:48.176294 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.176222    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92359562-7966-400d-87ed-d7d35b885d40-tmp\") pod \"kube-auth-proxy-98dc6956-pmkj5\" (UID: \"92359562-7966-400d-87ed-d7d35b885d40\") " pod="openshift-ingress/kube-auth-proxy-98dc6956-pmkj5"
Jul 06 17:49:48.187817 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.187794    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kq5r8"]
Jul 06 17:49:48.190427 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.190411    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kq5r8"
Jul 06 17:49:48.193169 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.193152    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-5trq6\""
Jul 06 17:49:48.193267 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.193194    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 06 17:49:48.199702 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.199680    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kq5r8"]
Jul 06 17:49:48.277085 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.277037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92359562-7966-400d-87ed-d7d35b885d40-tls-certs\") pod \"kube-auth-proxy-98dc6956-pmkj5\" (UID: \"92359562-7966-400d-87ed-d7d35b885d40\") " pod="openshift-ingress/kube-auth-proxy-98dc6956-pmkj5"
Jul 06 17:49:48.277085 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.277063    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x8jq\" (UniqueName: \"kubernetes.io/projected/92359562-7966-400d-87ed-d7d35b885d40-kube-api-access-7x8jq\") pod \"kube-auth-proxy-98dc6956-pmkj5\" (UID: \"92359562-7966-400d-87ed-d7d35b885d40\") " pod="openshift-ingress/kube-auth-proxy-98dc6956-pmkj5"
Jul 06 17:49:48.277085 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.277083    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92359562-7966-400d-87ed-d7d35b885d40-tmp\") pod \"kube-auth-proxy-98dc6956-pmkj5\" (UID: \"92359562-7966-400d-87ed-d7d35b885d40\") " pod="openshift-ingress/kube-auth-proxy-98dc6956-pmkj5"
Jul 06 17:49:48.279237 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.279211    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92359562-7966-400d-87ed-d7d35b885d40-tmp\") pod \"kube-auth-proxy-98dc6956-pmkj5\" (UID: \"92359562-7966-400d-87ed-d7d35b885d40\") " pod="openshift-ingress/kube-auth-proxy-98dc6956-pmkj5"
Jul 06 17:49:48.279353 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.279337    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92359562-7966-400d-87ed-d7d35b885d40-tls-certs\") pod \"kube-auth-proxy-98dc6956-pmkj5\" (UID: \"92359562-7966-400d-87ed-d7d35b885d40\") " pod="openshift-ingress/kube-auth-proxy-98dc6956-pmkj5"
Jul 06 17:49:48.284189 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.284167    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x8jq\" (UniqueName: \"kubernetes.io/projected/92359562-7966-400d-87ed-d7d35b885d40-kube-api-access-7x8jq\") pod \"kube-auth-proxy-98dc6956-pmkj5\" (UID: \"92359562-7966-400d-87ed-d7d35b885d40\") " pod="openshift-ingress/kube-auth-proxy-98dc6956-pmkj5"
Jul 06 17:49:48.304978 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.304957    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-98dc6956-pmkj5"
Jul 06 17:49:48.378120 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.378091    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68k99\" (UniqueName: \"kubernetes.io/projected/16e03d15-556c-4eaf-8f49-e5210b9422bc-kube-api-access-68k99\") pod \"odh-model-controller-7998c67dc4-kq5r8\" (UID: \"16e03d15-556c-4eaf-8f49-e5210b9422bc\") " pod="opendatahub/odh-model-controller-7998c67dc4-kq5r8"
Jul 06 17:49:48.378265 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.378168    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16e03d15-556c-4eaf-8f49-e5210b9422bc-cert\") pod \"odh-model-controller-7998c67dc4-kq5r8\" (UID: \"16e03d15-556c-4eaf-8f49-e5210b9422bc\") " pod="opendatahub/odh-model-controller-7998c67dc4-kq5r8"
Jul 06 17:49:48.422451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.422430    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-98dc6956-pmkj5"]
Jul 06 17:49:48.479287 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.479258    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16e03d15-556c-4eaf-8f49-e5210b9422bc-cert\") pod \"odh-model-controller-7998c67dc4-kq5r8\" (UID: \"16e03d15-556c-4eaf-8f49-e5210b9422bc\") " pod="opendatahub/odh-model-controller-7998c67dc4-kq5r8"
Jul 06 17:49:48.479403 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.479309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68k99\" (UniqueName: \"kubernetes.io/projected/16e03d15-556c-4eaf-8f49-e5210b9422bc-kube-api-access-68k99\") pod \"odh-model-controller-7998c67dc4-kq5r8\" (UID: \"16e03d15-556c-4eaf-8f49-e5210b9422bc\") " pod="opendatahub/odh-model-controller-7998c67dc4-kq5r8"
Jul 06 17:49:48.481442 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.481423    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16e03d15-556c-4eaf-8f49-e5210b9422bc-cert\") pod \"odh-model-controller-7998c67dc4-kq5r8\" (UID: \"16e03d15-556c-4eaf-8f49-e5210b9422bc\") " pod="opendatahub/odh-model-controller-7998c67dc4-kq5r8"
Jul 06 17:49:48.487053 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.487029    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68k99\" (UniqueName: \"kubernetes.io/projected/16e03d15-556c-4eaf-8f49-e5210b9422bc-kube-api-access-68k99\") pod \"odh-model-controller-7998c67dc4-kq5r8\" (UID: \"16e03d15-556c-4eaf-8f49-e5210b9422bc\") " pod="opendatahub/odh-model-controller-7998c67dc4-kq5r8"
Jul 06 17:49:48.499801 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.499784    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kq5r8"
Jul 06 17:49:48.618582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.618528    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kq5r8"]
Jul 06 17:49:48.620637 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:49:48.620609    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e03d15_556c_4eaf_8f49_e5210b9422bc.slice/crio-d5c2510efc7c284f4346146517914f21ee19a728c2ad004d44dc69149dabf9ce WatchSource:0}: Error finding container d5c2510efc7c284f4346146517914f21ee19a728c2ad004d44dc69149dabf9ce: Status 404 returned error can't find the container with id d5c2510efc7c284f4346146517914f21ee19a728c2ad004d44dc69149dabf9ce
Jul 06 17:49:48.886322 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.886254    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kq5r8" event={"ID":"16e03d15-556c-4eaf-8f49-e5210b9422bc","Type":"ContainerStarted","Data":"d5c2510efc7c284f4346146517914f21ee19a728c2ad004d44dc69149dabf9ce"}
Jul 06 17:49:48.887413 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:48.887385    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-98dc6956-pmkj5" event={"ID":"92359562-7966-400d-87ed-d7d35b885d40","Type":"ContainerStarted","Data":"a9f1d7ec5d91f2207a58764e920536c1dfc7d0fbae1f44f015cbe87af54d9f57"}
Jul 06 17:49:49.089541 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.089512    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"]
Jul 06 17:49:49.092816 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.092794    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.095351 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.095325    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 06 17:49:49.095531 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.095510    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 06 17:49:49.095781 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.095764    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 06 17:49:49.096104 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.096083    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 17:49:49.096104 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.096098    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gvj69\""
Jul 06 17:49:49.106039 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.106006    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"]
Jul 06 17:49:49.284776 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.284742    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/87507934-5547-4a6c-a196-ac0c37d188a5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.285025 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.284810    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/87507934-5547-4a6c-a196-ac0c37d188a5-local-certs\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.285025 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.284835    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-cacerts\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.285025 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.284860    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87507934-5547-4a6c-a196-ac0c37d188a5-istio-token\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.285025 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.284936    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.285025 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.284972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.285025 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.284992    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgx7t\" (UniqueName: \"kubernetes.io/projected/87507934-5547-4a6c-a196-ac0c37d188a5-kube-api-access-qgx7t\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.386120 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.386093    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.386259 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.386142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.386259 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.386171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgx7t\" (UniqueName: \"kubernetes.io/projected/87507934-5547-4a6c-a196-ac0c37d188a5-kube-api-access-qgx7t\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.386259 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.386206    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/87507934-5547-4a6c-a196-ac0c37d188a5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.386259 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.386245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/87507934-5547-4a6c-a196-ac0c37d188a5-local-certs\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.386464 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.386265    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-cacerts\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.386464 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.386295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87507934-5547-4a6c-a196-ac0c37d188a5-istio-token\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.387414 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.387354    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/87507934-5547-4a6c-a196-ac0c37d188a5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.389374 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.389346    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.389711 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.389552    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.390046 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.390024    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/87507934-5547-4a6c-a196-ac0c37d188a5-local-certs\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.390178 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.390156    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-cacerts\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.394529 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.394508    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87507934-5547-4a6c-a196-ac0c37d188a5-istio-token\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.394803 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.394781    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgx7t\" (UniqueName: \"kubernetes.io/projected/87507934-5547-4a6c-a196-ac0c37d188a5-kube-api-access-qgx7t\") pod \"istiod-openshift-gateway-7574459656-8j67b\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.405549 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.405428    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:49.551466 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.551433    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"]
Jul 06 17:49:49.553666 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:49:49.553642    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87507934_5547_4a6c_a196_ac0c37d188a5.slice/crio-b5f8894775b3e397c7867ed0fc0c93fc62f51ed4bd7ff84a847861992795481f WatchSource:0}: Error finding container b5f8894775b3e397c7867ed0fc0c93fc62f51ed4bd7ff84a847861992795481f: Status 404 returned error can't find the container with id b5f8894775b3e397c7867ed0fc0c93fc62f51ed4bd7ff84a847861992795481f
Jul 06 17:49:49.893227 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:49.893149    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b" event={"ID":"87507934-5547-4a6c-a196-ac0c37d188a5","Type":"ContainerStarted","Data":"b5f8894775b3e397c7867ed0fc0c93fc62f51ed4bd7ff84a847861992795481f"}
Jul 06 17:49:53.449519 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:53.449484    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 06 17:49:53.449867 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:53.449561    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 06 17:49:53.912874 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:53.912837    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-98dc6956-pmkj5" event={"ID":"92359562-7966-400d-87ed-d7d35b885d40","Type":"ContainerStarted","Data":"d8493aa6ab2198a9287ee301015a626025f65fb34d80f197b518c0d78f7c9f8b"}
Jul 06 17:49:53.914461 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:53.914435    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b" event={"ID":"87507934-5547-4a6c-a196-ac0c37d188a5","Type":"ContainerStarted","Data":"bdf6f1186838b753db8364ef2ece14b86941cd744483934def331c1de402001e"}
Jul 06 17:49:53.914619 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:53.914490    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:53.915820 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:53.915794    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kq5r8" event={"ID":"16e03d15-556c-4eaf-8f49-e5210b9422bc","Type":"ContainerStarted","Data":"5b01c9b7796aeccbce5b6ae39a023f5458a8551d21f4fcc8f6cea2591b97385d"}
Jul 06 17:49:53.915958 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:53.915947    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-kq5r8"
Jul 06 17:49:53.928334 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:53.928288    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-98dc6956-pmkj5" podStartSLOduration=1.9067405179999999 podStartE2EDuration="6.928271869s" totalImagesPullingTime="5.021531351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:49:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:49:53.927498179 +0000 UTC m=+569.520952723" watchObservedRunningTime="2026-07-06 17:49:53.928271869 +0000 UTC m=+569.521726399"
Jul 06 17:49:53.945224 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:53.945102    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b" podStartSLOduration=1.052030444 podStartE2EDuration="4.945071884s" totalImagesPullingTime="3.89304144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:49:53.944696039 +0000 UTC m=+569.538150590" watchObservedRunningTime="2026-07-06 17:49:53.945071884 +0000 UTC m=+569.538526414"
Jul 06 17:49:54.921424 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:54.921396    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:49:54.941374 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:54.941332    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-kq5r8" podStartSLOduration=2.117405491 podStartE2EDuration="6.941317902s" totalImagesPullingTime="4.823912411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:49:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:49:53.961875225 +0000 UTC m=+569.555329754" watchObservedRunningTime="2026-07-06 17:49:54.941317902 +0000 UTC m=+570.534772434"
Jul 06 17:49:58.692617 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.692586    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb"]
Jul 06 17:49:58.696054 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.696038    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb"
Jul 06 17:49:58.699268 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.699251    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 06 17:49:58.699407 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.699392    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-nj8tz\""
Jul 06 17:49:58.705012 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.704992    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr"]
Jul 06 17:49:58.707965 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.707948    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr"
Jul 06 17:49:58.711972 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.711953    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 06 17:49:58.712058 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.712042    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-vjfms\""
Jul 06 17:49:58.715092 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.715071    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb"]
Jul 06 17:49:58.729334 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.729312    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr"]
Jul 06 17:49:58.758190 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.757465    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e76a1ea-7310-49a7-a735-6556f2add542-cert\") pod \"kserve-controller-manager-d6f4b65b8-2ngwb\" (UID: \"1e76a1ea-7310-49a7-a735-6556f2add542\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb"
Jul 06 17:49:58.758190 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.757509    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d6517d6-c2c5-49ca-abce-dfc7a435deeb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f42lr\" (UID: \"0d6517d6-c2c5-49ca-abce-dfc7a435deeb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr"
Jul 06 17:49:58.758190 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.757549    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bww9k\" (UniqueName: \"kubernetes.io/projected/1e76a1ea-7310-49a7-a735-6556f2add542-kube-api-access-bww9k\") pod \"kserve-controller-manager-d6f4b65b8-2ngwb\" (UID: \"1e76a1ea-7310-49a7-a735-6556f2add542\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb"
Jul 06 17:49:58.758190 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.757626    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r8vc\" (UniqueName: \"kubernetes.io/projected/0d6517d6-c2c5-49ca-abce-dfc7a435deeb-kube-api-access-9r8vc\") pod \"llmisvc-controller-manager-777c6bd7d5-f42lr\" (UID: \"0d6517d6-c2c5-49ca-abce-dfc7a435deeb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr"
Jul 06 17:49:58.858585 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.858562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r8vc\" (UniqueName: \"kubernetes.io/projected/0d6517d6-c2c5-49ca-abce-dfc7a435deeb-kube-api-access-9r8vc\") pod \"llmisvc-controller-manager-777c6bd7d5-f42lr\" (UID: \"0d6517d6-c2c5-49ca-abce-dfc7a435deeb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr"
Jul 06 17:49:58.858686 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.858598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e76a1ea-7310-49a7-a735-6556f2add542-cert\") pod \"kserve-controller-manager-d6f4b65b8-2ngwb\" (UID: \"1e76a1ea-7310-49a7-a735-6556f2add542\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb"
Jul 06 17:49:58.858686 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.858613    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d6517d6-c2c5-49ca-abce-dfc7a435deeb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f42lr\" (UID: \"0d6517d6-c2c5-49ca-abce-dfc7a435deeb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr"
Jul 06 17:49:58.858686 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.858655    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bww9k\" (UniqueName: \"kubernetes.io/projected/1e76a1ea-7310-49a7-a735-6556f2add542-kube-api-access-bww9k\") pod \"kserve-controller-manager-d6f4b65b8-2ngwb\" (UID: \"1e76a1ea-7310-49a7-a735-6556f2add542\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb"
Jul 06 17:49:58.858802 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:49:58.858687    2581 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 06 17:49:58.858802 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:49:58.858710    2581 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 06 17:49:58.858802 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:49:58.858741    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d6517d6-c2c5-49ca-abce-dfc7a435deeb-cert podName:0d6517d6-c2c5-49ca-abce-dfc7a435deeb nodeName:}" failed. No retries permitted until 2026-07-06 17:49:59.358723985 +0000 UTC m=+574.952178497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0d6517d6-c2c5-49ca-abce-dfc7a435deeb-cert") pod "llmisvc-controller-manager-777c6bd7d5-f42lr" (UID: "0d6517d6-c2c5-49ca-abce-dfc7a435deeb") : secret "llmisvc-webhook-server-cert" not found
Jul 06 17:49:58.858802 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:49:58.858761    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e76a1ea-7310-49a7-a735-6556f2add542-cert podName:1e76a1ea-7310-49a7-a735-6556f2add542 nodeName:}" failed. No retries permitted until 2026-07-06 17:49:59.358750151 +0000 UTC m=+574.952204658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1e76a1ea-7310-49a7-a735-6556f2add542-cert") pod "kserve-controller-manager-d6f4b65b8-2ngwb" (UID: "1e76a1ea-7310-49a7-a735-6556f2add542") : secret "kserve-webhook-server-cert" not found
Jul 06 17:49:58.871743 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.871713    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r8vc\" (UniqueName: \"kubernetes.io/projected/0d6517d6-c2c5-49ca-abce-dfc7a435deeb-kube-api-access-9r8vc\") pod \"llmisvc-controller-manager-777c6bd7d5-f42lr\" (UID: \"0d6517d6-c2c5-49ca-abce-dfc7a435deeb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr"
Jul 06 17:49:58.871969 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:58.871952    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bww9k\" (UniqueName: \"kubernetes.io/projected/1e76a1ea-7310-49a7-a735-6556f2add542-kube-api-access-bww9k\") pod \"kserve-controller-manager-d6f4b65b8-2ngwb\" (UID: \"1e76a1ea-7310-49a7-a735-6556f2add542\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb"
Jul 06 17:49:59.361761 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:59.361729    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e76a1ea-7310-49a7-a735-6556f2add542-cert\") pod \"kserve-controller-manager-d6f4b65b8-2ngwb\" (UID: \"1e76a1ea-7310-49a7-a735-6556f2add542\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb"
Jul 06 17:49:59.361761 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:59.361763    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d6517d6-c2c5-49ca-abce-dfc7a435deeb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f42lr\" (UID: \"0d6517d6-c2c5-49ca-abce-dfc7a435deeb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr"
Jul 06 17:49:59.363877 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:59.363856    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d6517d6-c2c5-49ca-abce-dfc7a435deeb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f42lr\" (UID: \"0d6517d6-c2c5-49ca-abce-dfc7a435deeb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr"
Jul 06 17:49:59.363999 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:59.363945    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e76a1ea-7310-49a7-a735-6556f2add542-cert\") pod \"kserve-controller-manager-d6f4b65b8-2ngwb\" (UID: \"1e76a1ea-7310-49a7-a735-6556f2add542\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb"
Jul 06 17:49:59.605608 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:59.605582    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb"
Jul 06 17:49:59.616737 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:59.616720    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr"
Jul 06 17:49:59.761698 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:59.761674    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb"]
Jul 06 17:49:59.763766 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:49:59.763743    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e76a1ea_7310_49a7_a735_6556f2add542.slice/crio-f5606c148a84e5b802f3f32360da754699b7c3bb7dee96d52bba917294077619 WatchSource:0}: Error finding container f5606c148a84e5b802f3f32360da754699b7c3bb7dee96d52bba917294077619: Status 404 returned error can't find the container with id f5606c148a84e5b802f3f32360da754699b7c3bb7dee96d52bba917294077619
Jul 06 17:49:59.771965 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:59.771941    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr"]
Jul 06 17:49:59.773192 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:49:59.773172    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0d6517d6_c2c5_49ca_abce_dfc7a435deeb.slice/crio-bbfd8a04cf4ca240e640b139605991f47cf40e5e77e7a05b996e54dc682ebb2e WatchSource:0}: Error finding container bbfd8a04cf4ca240e640b139605991f47cf40e5e77e7a05b996e54dc682ebb2e: Status 404 returned error can't find the container with id bbfd8a04cf4ca240e640b139605991f47cf40e5e77e7a05b996e54dc682ebb2e
Jul 06 17:49:59.937340 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:59.937308    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb" event={"ID":"1e76a1ea-7310-49a7-a735-6556f2add542","Type":"ContainerStarted","Data":"f5606c148a84e5b802f3f32360da754699b7c3bb7dee96d52bba917294077619"}
Jul 06 17:49:59.938341 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:49:59.938318    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr" event={"ID":"0d6517d6-c2c5-49ca-abce-dfc7a435deeb","Type":"ContainerStarted","Data":"bbfd8a04cf4ca240e640b139605991f47cf40e5e77e7a05b996e54dc682ebb2e"}
Jul 06 17:50:02.272638 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.272604    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"]
Jul 06 17:50:02.273026 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.272787    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b" podUID="87507934-5547-4a6c-a196-ac0c37d188a5" containerName="discovery" containerID="cri-o://bdf6f1186838b753db8364ef2ece14b86941cd744483934def331c1de402001e" gracePeriod=30
Jul 06 17:50:02.519319 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.519300    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:50:02.585312 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.585243    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-cacerts\") pod \"87507934-5547-4a6c-a196-ac0c37d188a5\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") "
Jul 06 17:50:02.585428 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.585326    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/87507934-5547-4a6c-a196-ac0c37d188a5-istio-csr-ca-configmap\") pod \"87507934-5547-4a6c-a196-ac0c37d188a5\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") "
Jul 06 17:50:02.585428 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.585370    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-istio-csr-dns-cert\") pod \"87507934-5547-4a6c-a196-ac0c37d188a5\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") "
Jul 06 17:50:02.585577 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.585553    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87507934-5547-4a6c-a196-ac0c37d188a5-istio-token\") pod \"87507934-5547-4a6c-a196-ac0c37d188a5\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") "
Jul 06 17:50:02.585643 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.585603    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgx7t\" (UniqueName: \"kubernetes.io/projected/87507934-5547-4a6c-a196-ac0c37d188a5-kube-api-access-qgx7t\") pod \"87507934-5547-4a6c-a196-ac0c37d188a5\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") "
Jul 06 17:50:02.585643 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.585635    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-istio-kubeconfig\") pod \"87507934-5547-4a6c-a196-ac0c37d188a5\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") "
Jul 06 17:50:02.585744 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.585699    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/87507934-5547-4a6c-a196-ac0c37d188a5-local-certs\") pod \"87507934-5547-4a6c-a196-ac0c37d188a5\" (UID: \"87507934-5547-4a6c-a196-ac0c37d188a5\") "
Jul 06 17:50:02.585798 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.585776    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87507934-5547-4a6c-a196-ac0c37d188a5-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "87507934-5547-4a6c-a196-ac0c37d188a5" (UID: "87507934-5547-4a6c-a196-ac0c37d188a5"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 17:50:02.586068 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.586046    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/87507934-5547-4a6c-a196-ac0c37d188a5-istio-csr-ca-configmap\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:50:02.588383 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.588336    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-cacerts" (OuterVolumeSpecName: "cacerts") pod "87507934-5547-4a6c-a196-ac0c37d188a5" (UID: "87507934-5547-4a6c-a196-ac0c37d188a5"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 17:50:02.588609 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.588577    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87507934-5547-4a6c-a196-ac0c37d188a5-istio-token" (OuterVolumeSpecName: "istio-token") pod "87507934-5547-4a6c-a196-ac0c37d188a5" (UID: "87507934-5547-4a6c-a196-ac0c37d188a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:50:02.588788 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.588756    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87507934-5547-4a6c-a196-ac0c37d188a5-kube-api-access-qgx7t" (OuterVolumeSpecName: "kube-api-access-qgx7t") pod "87507934-5547-4a6c-a196-ac0c37d188a5" (UID: "87507934-5547-4a6c-a196-ac0c37d188a5"). InnerVolumeSpecName "kube-api-access-qgx7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:50:02.589140 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.589107    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "87507934-5547-4a6c-a196-ac0c37d188a5" (UID: "87507934-5547-4a6c-a196-ac0c37d188a5"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 17:50:02.589288 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.589256    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87507934-5547-4a6c-a196-ac0c37d188a5-local-certs" (OuterVolumeSpecName: "local-certs") pod "87507934-5547-4a6c-a196-ac0c37d188a5" (UID: "87507934-5547-4a6c-a196-ac0c37d188a5"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 17:50:02.589885 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.589860    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "87507934-5547-4a6c-a196-ac0c37d188a5" (UID: "87507934-5547-4a6c-a196-ac0c37d188a5"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 17:50:02.686557 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.686533    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87507934-5547-4a6c-a196-ac0c37d188a5-istio-token\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:50:02.686557 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.686553    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgx7t\" (UniqueName: \"kubernetes.io/projected/87507934-5547-4a6c-a196-ac0c37d188a5-kube-api-access-qgx7t\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:50:02.686674 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.686561    2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-istio-kubeconfig\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:50:02.686674 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.686569    2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/87507934-5547-4a6c-a196-ac0c37d188a5-local-certs\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:50:02.686674 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.686577    2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-cacerts\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:50:02.686674 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.686585    2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/87507934-5547-4a6c-a196-ac0c37d188a5-istio-csr-dns-cert\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:50:02.951975 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.951949    2581 generic.go:356] "Generic (PLEG): container finished" podID="87507934-5547-4a6c-a196-ac0c37d188a5" containerID="bdf6f1186838b753db8364ef2ece14b86941cd744483934def331c1de402001e" exitCode=0
Jul 06 17:50:02.952058 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.951997    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"
Jul 06 17:50:02.952058 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.952036    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b" event={"ID":"87507934-5547-4a6c-a196-ac0c37d188a5","Type":"ContainerDied","Data":"bdf6f1186838b753db8364ef2ece14b86941cd744483934def331c1de402001e"}
Jul 06 17:50:02.952128 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.952084    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8j67b" event={"ID":"87507934-5547-4a6c-a196-ac0c37d188a5","Type":"ContainerDied","Data":"b5f8894775b3e397c7867ed0fc0c93fc62f51ed4bd7ff84a847861992795481f"}
Jul 06 17:50:02.952128 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.952109    2581 scope.go:117] "RemoveContainer" containerID="bdf6f1186838b753db8364ef2ece14b86941cd744483934def331c1de402001e"
Jul 06 17:50:02.960043 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.960012    2581 scope.go:117] "RemoveContainer" containerID="bdf6f1186838b753db8364ef2ece14b86941cd744483934def331c1de402001e"
Jul 06 17:50:02.960258 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:50:02.960242    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdf6f1186838b753db8364ef2ece14b86941cd744483934def331c1de402001e\": container with ID starting with bdf6f1186838b753db8364ef2ece14b86941cd744483934def331c1de402001e not found: ID does not exist" containerID="bdf6f1186838b753db8364ef2ece14b86941cd744483934def331c1de402001e"
Jul 06 17:50:02.960302 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.960266    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdf6f1186838b753db8364ef2ece14b86941cd744483934def331c1de402001e"} err="failed to get container status \"bdf6f1186838b753db8364ef2ece14b86941cd744483934def331c1de402001e\": rpc error: code = NotFound desc = could not find container \"bdf6f1186838b753db8364ef2ece14b86941cd744483934def331c1de402001e\": container with ID starting with bdf6f1186838b753db8364ef2ece14b86941cd744483934def331c1de402001e not found: ID does not exist"
Jul 06 17:50:02.968940 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.968885    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"]
Jul 06 17:50:02.979117 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:02.979093    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8j67b"]
Jul 06 17:50:04.911272 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:04.911242    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87507934-5547-4a6c-a196-ac0c37d188a5" path="/var/lib/kubelet/pods/87507934-5547-4a6c-a196-ac0c37d188a5/volumes"
Jul 06 17:50:04.921974 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:04.921885    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-kq5r8"
Jul 06 17:50:04.965341 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:04.965314    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb" event={"ID":"1e76a1ea-7310-49a7-a735-6556f2add542","Type":"ContainerStarted","Data":"97dbf3d46765a4c3d1010b67637b4d2853a8c7d9f12082c4778de912f8e59868"}
Jul 06 17:50:04.965585 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:04.965524    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb"
Jul 06 17:50:04.967326 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:04.967302    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr" event={"ID":"0d6517d6-c2c5-49ca-abce-dfc7a435deeb","Type":"ContainerStarted","Data":"ead8ddb47c48113d42b5847684791deeec530eac746662d624cc9dc55d1f47fa"}
Jul 06 17:50:04.967442 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:04.967411    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr"
Jul 06 17:50:04.981612 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:04.981575    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb" podStartSLOduration=2.5783594389999998 podStartE2EDuration="6.981561976s" totalImagesPullingTime="4.403202537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:50:04.980327802 +0000 UTC m=+580.573782355" watchObservedRunningTime="2026-07-06 17:50:04.981561976 +0000 UTC m=+580.575016846"
Jul 06 17:50:04.996009 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:04.995972    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr" podStartSLOduration=2.601980515 podStartE2EDuration="6.99596368s" totalImagesPullingTime="4.393983165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:50:04.995405497 +0000 UTC m=+580.588860026" watchObservedRunningTime="2026-07-06 17:50:04.99596368 +0000 UTC m=+580.589418208"
Jul 06 17:50:35.973869 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:35.973825    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f42lr"
Jul 06 17:50:35.977263 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:35.976754    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ngwb"
Jul 06 17:50:49.024624 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.024544    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nb6xs"]
Jul 06 17:50:49.024985 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.024785    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87507934-5547-4a6c-a196-ac0c37d188a5" containerName="discovery"
Jul 06 17:50:49.024985 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.024796    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="87507934-5547-4a6c-a196-ac0c37d188a5" containerName="discovery"
Jul 06 17:50:49.024985 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.024849    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="87507934-5547-4a6c-a196-ac0c37d188a5" containerName="discovery"
Jul 06 17:50:49.027778 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.027754    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nb6xs"
Jul 06 17:50:49.030662 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.030636    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 17:50:49.031202 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.031178    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 17:50:49.031936 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.031914    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-vddcv\""
Jul 06 17:50:49.036291 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.036140    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nb6xs"]
Jul 06 17:50:49.109529 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.109505    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkcq7\" (UniqueName: \"kubernetes.io/projected/8bbc90f4-3954-40da-8ff1-c0dff4f39745-kube-api-access-wkcq7\") pod \"kuadrant-operator-catalog-nb6xs\" (UID: \"8bbc90f4-3954-40da-8ff1-c0dff4f39745\") " pod="kuadrant-system/kuadrant-operator-catalog-nb6xs"
Jul 06 17:50:49.210607 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.210580    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkcq7\" (UniqueName: \"kubernetes.io/projected/8bbc90f4-3954-40da-8ff1-c0dff4f39745-kube-api-access-wkcq7\") pod \"kuadrant-operator-catalog-nb6xs\" (UID: \"8bbc90f4-3954-40da-8ff1-c0dff4f39745\") " pod="kuadrant-system/kuadrant-operator-catalog-nb6xs"
Jul 06 17:50:49.218753 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.218726    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkcq7\" (UniqueName: \"kubernetes.io/projected/8bbc90f4-3954-40da-8ff1-c0dff4f39745-kube-api-access-wkcq7\") pod \"kuadrant-operator-catalog-nb6xs\" (UID: \"8bbc90f4-3954-40da-8ff1-c0dff4f39745\") " pod="kuadrant-system/kuadrant-operator-catalog-nb6xs"
Jul 06 17:50:49.338874 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.338809    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nb6xs"
Jul 06 17:50:49.389605 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.389572    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nb6xs"]
Jul 06 17:50:49.456647 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.456619    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nb6xs"]
Jul 06 17:50:49.458945 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:50:49.458921    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bbc90f4_3954_40da_8ff1_c0dff4f39745.slice/crio-993ad1eec9e935d18fe0a49082b955fd5a967665ab98ec8a2665fb68c14687b7 WatchSource:0}: Error finding container 993ad1eec9e935d18fe0a49082b955fd5a967665ab98ec8a2665fb68c14687b7: Status 404 returned error can't find the container with id 993ad1eec9e935d18fe0a49082b955fd5a967665ab98ec8a2665fb68c14687b7
Jul 06 17:50:49.597211 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.597159    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2wnrn"]
Jul 06 17:50:49.601092 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.601075    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2wnrn"
Jul 06 17:50:49.607989 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.607703    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2wnrn"]
Jul 06 17:50:49.715365 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.715342    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-288b5\" (UniqueName: \"kubernetes.io/projected/e6b10695-d2ce-417d-8d5f-32ebb6ed3223-kube-api-access-288b5\") pod \"kuadrant-operator-catalog-2wnrn\" (UID: \"e6b10695-d2ce-417d-8d5f-32ebb6ed3223\") " pod="kuadrant-system/kuadrant-operator-catalog-2wnrn"
Jul 06 17:50:49.815797 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.815755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288b5\" (UniqueName: \"kubernetes.io/projected/e6b10695-d2ce-417d-8d5f-32ebb6ed3223-kube-api-access-288b5\") pod \"kuadrant-operator-catalog-2wnrn\" (UID: \"e6b10695-d2ce-417d-8d5f-32ebb6ed3223\") " pod="kuadrant-system/kuadrant-operator-catalog-2wnrn"
Jul 06 17:50:49.824192 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.824173    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-288b5\" (UniqueName: \"kubernetes.io/projected/e6b10695-d2ce-417d-8d5f-32ebb6ed3223-kube-api-access-288b5\") pod \"kuadrant-operator-catalog-2wnrn\" (UID: \"e6b10695-d2ce-417d-8d5f-32ebb6ed3223\") " pod="kuadrant-system/kuadrant-operator-catalog-2wnrn"
Jul 06 17:50:49.910627 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:49.910563    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2wnrn"
Jul 06 17:50:50.035532 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:50.035508    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2wnrn"]
Jul 06 17:50:50.038085 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:50:50.038059    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b10695_d2ce_417d_8d5f_32ebb6ed3223.slice/crio-b8f45cdab3dd308c2bc0401c7520df88b2c5fd5d16f66d653a3f5c9d5912d8fe WatchSource:0}: Error finding container b8f45cdab3dd308c2bc0401c7520df88b2c5fd5d16f66d653a3f5c9d5912d8fe: Status 404 returned error can't find the container with id b8f45cdab3dd308c2bc0401c7520df88b2c5fd5d16f66d653a3f5c9d5912d8fe
Jul 06 17:50:50.136212 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:50.136180    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nb6xs" event={"ID":"8bbc90f4-3954-40da-8ff1-c0dff4f39745","Type":"ContainerStarted","Data":"993ad1eec9e935d18fe0a49082b955fd5a967665ab98ec8a2665fb68c14687b7"}
Jul 06 17:50:50.137301 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:50.137279    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2wnrn" event={"ID":"e6b10695-d2ce-417d-8d5f-32ebb6ed3223","Type":"ContainerStarted","Data":"b8f45cdab3dd308c2bc0401c7520df88b2c5fd5d16f66d653a3f5c9d5912d8fe"}
Jul 06 17:50:52.148131 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:52.148100    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2wnrn" event={"ID":"e6b10695-d2ce-417d-8d5f-32ebb6ed3223","Type":"ContainerStarted","Data":"a35906f62638fddbc37100fd6018fd812ada535a851ad72cb325ca9a77a1be9a"}
Jul 06 17:50:52.150131 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:52.149938    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-nb6xs" podUID="8bbc90f4-3954-40da-8ff1-c0dff4f39745" containerName="registry-server" containerID="cri-o://cc2829fe4224b4aabbc46f7e4e543525bc1e17e191fc946a77c5e644a184f68f" gracePeriod=2
Jul 06 17:50:52.153906 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:52.149760    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nb6xs" event={"ID":"8bbc90f4-3954-40da-8ff1-c0dff4f39745","Type":"ContainerStarted","Data":"cc2829fe4224b4aabbc46f7e4e543525bc1e17e191fc946a77c5e644a184f68f"}
Jul 06 17:50:52.165685 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:52.165221    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2wnrn" podStartSLOduration=1.28849811 podStartE2EDuration="3.165207173s" totalImagesPullingTime="1.876709063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:50:52.165056999 +0000 UTC m=+627.758511527" watchObservedRunningTime="2026-07-06 17:50:52.165207173 +0000 UTC m=+627.758661702"
Jul 06 17:50:52.180323 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:52.180282    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-nb6xs" podStartSLOduration=0.727056217 podStartE2EDuration="3.180270458s" totalImagesPullingTime="2.453214241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:50:52.178675671 +0000 UTC m=+627.772130211" watchObservedRunningTime="2026-07-06 17:50:52.180270458 +0000 UTC m=+627.773724985"
Jul 06 17:50:52.388360 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:52.388337    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nb6xs"
Jul 06 17:50:52.534541 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:52.534501    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkcq7\" (UniqueName: \"kubernetes.io/projected/8bbc90f4-3954-40da-8ff1-c0dff4f39745-kube-api-access-wkcq7\") pod \"8bbc90f4-3954-40da-8ff1-c0dff4f39745\" (UID: \"8bbc90f4-3954-40da-8ff1-c0dff4f39745\") "
Jul 06 17:50:52.536531 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:52.536507    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bbc90f4-3954-40da-8ff1-c0dff4f39745-kube-api-access-wkcq7" (OuterVolumeSpecName: "kube-api-access-wkcq7") pod "8bbc90f4-3954-40da-8ff1-c0dff4f39745" (UID: "8bbc90f4-3954-40da-8ff1-c0dff4f39745"). InnerVolumeSpecName "kube-api-access-wkcq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:50:52.635531 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:52.635494    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkcq7\" (UniqueName: \"kubernetes.io/projected/8bbc90f4-3954-40da-8ff1-c0dff4f39745-kube-api-access-wkcq7\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:50:53.155774 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:53.155744    2581 generic.go:356] "Generic (PLEG): container finished" podID="8bbc90f4-3954-40da-8ff1-c0dff4f39745" containerID="cc2829fe4224b4aabbc46f7e4e543525bc1e17e191fc946a77c5e644a184f68f" exitCode=0
Jul 06 17:50:53.156136 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:53.155808    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nb6xs"
Jul 06 17:50:53.156136 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:53.155835    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nb6xs" event={"ID":"8bbc90f4-3954-40da-8ff1-c0dff4f39745","Type":"ContainerDied","Data":"cc2829fe4224b4aabbc46f7e4e543525bc1e17e191fc946a77c5e644a184f68f"}
Jul 06 17:50:53.156136 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:53.155889    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nb6xs" event={"ID":"8bbc90f4-3954-40da-8ff1-c0dff4f39745","Type":"ContainerDied","Data":"993ad1eec9e935d18fe0a49082b955fd5a967665ab98ec8a2665fb68c14687b7"}
Jul 06 17:50:53.156136 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:53.155942    2581 scope.go:117] "RemoveContainer" containerID="cc2829fe4224b4aabbc46f7e4e543525bc1e17e191fc946a77c5e644a184f68f"
Jul 06 17:50:53.164514 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:53.164497    2581 scope.go:117] "RemoveContainer" containerID="cc2829fe4224b4aabbc46f7e4e543525bc1e17e191fc946a77c5e644a184f68f"
Jul 06 17:50:53.164750 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:50:53.164734    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc2829fe4224b4aabbc46f7e4e543525bc1e17e191fc946a77c5e644a184f68f\": container with ID starting with cc2829fe4224b4aabbc46f7e4e543525bc1e17e191fc946a77c5e644a184f68f not found: ID does not exist" containerID="cc2829fe4224b4aabbc46f7e4e543525bc1e17e191fc946a77c5e644a184f68f"
Jul 06 17:50:53.164803 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:53.164758    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc2829fe4224b4aabbc46f7e4e543525bc1e17e191fc946a77c5e644a184f68f"} err="failed to get container status \"cc2829fe4224b4aabbc46f7e4e543525bc1e17e191fc946a77c5e644a184f68f\": rpc error: code = NotFound desc = could not find container \"cc2829fe4224b4aabbc46f7e4e543525bc1e17e191fc946a77c5e644a184f68f\": container with ID starting with cc2829fe4224b4aabbc46f7e4e543525bc1e17e191fc946a77c5e644a184f68f not found: ID does not exist"
Jul 06 17:50:53.170999 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:53.170947    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nb6xs"]
Jul 06 17:50:53.174921 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:53.174882    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nb6xs"]
Jul 06 17:50:54.910814 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:54.910788    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bbc90f4-3954-40da-8ff1-c0dff4f39745" path="/var/lib/kubelet/pods/8bbc90f4-3954-40da-8ff1-c0dff4f39745/volumes"
Jul 06 17:50:59.911571 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:59.911534    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-2wnrn"
Jul 06 17:50:59.912060 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:59.911590    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-2wnrn"
Jul 06 17:50:59.933228 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:50:59.933205    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-2wnrn"
Jul 06 17:51:00.206995 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:00.206971    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-2wnrn"
Jul 06 17:51:15.503563 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:15.503530    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9t5th"]
Jul 06 17:51:15.503934 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:15.503785    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bbc90f4-3954-40da-8ff1-c0dff4f39745" containerName="registry-server"
Jul 06 17:51:15.503934 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:15.503796    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bbc90f4-3954-40da-8ff1-c0dff4f39745" containerName="registry-server"
Jul 06 17:51:15.503934 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:15.503863    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bbc90f4-3954-40da-8ff1-c0dff4f39745" containerName="registry-server"
Jul 06 17:51:15.508025 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:15.508008    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9t5th"
Jul 06 17:51:15.510946 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:15.510923    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-d8lcm\""
Jul 06 17:51:15.519468 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:15.519446    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9t5th"]
Jul 06 17:51:15.682473 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:15.682448    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxgdx\" (UniqueName: \"kubernetes.io/projected/6efca74b-5e60-4f88-8308-7cdac638cc62-kube-api-access-vxgdx\") pod \"authorino-operator-75cc8d5cb7-9t5th\" (UID: \"6efca74b-5e60-4f88-8308-7cdac638cc62\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9t5th"
Jul 06 17:51:15.783080 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:15.783013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxgdx\" (UniqueName: \"kubernetes.io/projected/6efca74b-5e60-4f88-8308-7cdac638cc62-kube-api-access-vxgdx\") pod \"authorino-operator-75cc8d5cb7-9t5th\" (UID: \"6efca74b-5e60-4f88-8308-7cdac638cc62\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9t5th"
Jul 06 17:51:15.795929 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:15.795188    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxgdx\" (UniqueName: \"kubernetes.io/projected/6efca74b-5e60-4f88-8308-7cdac638cc62-kube-api-access-vxgdx\") pod \"authorino-operator-75cc8d5cb7-9t5th\" (UID: \"6efca74b-5e60-4f88-8308-7cdac638cc62\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9t5th"
Jul 06 17:51:15.816970 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:15.816947    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9t5th"
Jul 06 17:51:15.940461 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:15.940316    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9t5th"]
Jul 06 17:51:15.943143 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:51:15.943120    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6efca74b_5e60_4f88_8308_7cdac638cc62.slice/crio-69d64196ad7ea3224db1e67e07b239a1ac74e58559c3844804ac0de5cec60449 WatchSource:0}: Error finding container 69d64196ad7ea3224db1e67e07b239a1ac74e58559c3844804ac0de5cec60449: Status 404 returned error can't find the container with id 69d64196ad7ea3224db1e67e07b239a1ac74e58559c3844804ac0de5cec60449
Jul 06 17:51:16.249160 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:16.249125    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9t5th" event={"ID":"6efca74b-5e60-4f88-8308-7cdac638cc62","Type":"ContainerStarted","Data":"69d64196ad7ea3224db1e67e07b239a1ac74e58559c3844804ac0de5cec60449"}
Jul 06 17:51:18.259462 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:18.259436    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9t5th" event={"ID":"6efca74b-5e60-4f88-8308-7cdac638cc62","Type":"ContainerStarted","Data":"e6871d29e677f6e3fa2d7a0d751664e06a8f02781c10893d46e84e47adc8f66c"}
Jul 06 17:51:18.259764 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:18.259557    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9t5th"
Jul 06 17:51:18.275026 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:18.274989    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9t5th" podStartSLOduration=1.021510888 podStartE2EDuration="3.27497569s" totalImagesPullingTime="2.253464802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:51:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:51:18.273999246 +0000 UTC m=+653.867453774" watchObservedRunningTime="2026-07-06 17:51:18.27497569 +0000 UTC m=+653.868430217"
Jul 06 17:51:18.742522 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:18.742492    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"]
Jul 06 17:51:18.745849 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:18.745832    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:18.748252 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:18.748234    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-pvrk8\""
Jul 06 17:51:18.754214 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:18.754194    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"]
Jul 06 17:51:18.901829 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:18.901802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjd9h\" (UniqueName: \"kubernetes.io/projected/c493513b-5ab9-4c97-a647-56134ae1e36f-kube-api-access-sjd9h\") pod \"limitador-operator-controller-manager-859d7b78bd-x2p4f\" (UID: \"c493513b-5ab9-4c97-a647-56134ae1e36f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:19.003009 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:19.002943    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjd9h\" (UniqueName: \"kubernetes.io/projected/c493513b-5ab9-4c97-a647-56134ae1e36f-kube-api-access-sjd9h\") pod \"limitador-operator-controller-manager-859d7b78bd-x2p4f\" (UID: \"c493513b-5ab9-4c97-a647-56134ae1e36f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:19.015595 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:19.015569    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjd9h\" (UniqueName: \"kubernetes.io/projected/c493513b-5ab9-4c97-a647-56134ae1e36f-kube-api-access-sjd9h\") pod \"limitador-operator-controller-manager-859d7b78bd-x2p4f\" (UID: \"c493513b-5ab9-4c97-a647-56134ae1e36f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:19.056432 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:19.056409    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:19.193829 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:19.193801    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"]
Jul 06 17:51:19.196032 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:51:19.196007    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc493513b_5ab9_4c97_a647_56134ae1e36f.slice/crio-06cb52104e73503f9c772616c21ddf69eebae79c3473a1ec426aefa6c5e68ab7 WatchSource:0}: Error finding container 06cb52104e73503f9c772616c21ddf69eebae79c3473a1ec426aefa6c5e68ab7: Status 404 returned error can't find the container with id 06cb52104e73503f9c772616c21ddf69eebae79c3473a1ec426aefa6c5e68ab7
Jul 06 17:51:19.264226 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:19.264171    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f" event={"ID":"c493513b-5ab9-4c97-a647-56134ae1e36f","Type":"ContainerStarted","Data":"06cb52104e73503f9c772616c21ddf69eebae79c3473a1ec426aefa6c5e68ab7"}
Jul 06 17:51:21.156747 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:21.156724    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-vdf5l"]
Jul 06 17:51:21.159728 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:21.159713    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vdf5l"
Jul 06 17:51:21.162613 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:21.162597    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-whz8k\""
Jul 06 17:51:21.162688 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:21.162627    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 06 17:51:21.171804 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:21.171787    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-vdf5l"]
Jul 06 17:51:21.274232 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:21.274208    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f" event={"ID":"c493513b-5ab9-4c97-a647-56134ae1e36f","Type":"ContainerStarted","Data":"a38f5a319ce2ca24bbf802405d0ac410a99c5268ad133de12d4eebb84ccfd662"}
Jul 06 17:51:21.274360 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:21.274349    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:21.291800 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:21.291760    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f" podStartSLOduration=1.412433807 podStartE2EDuration="3.291748933s" totalImagesPullingTime="1.879315126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:51:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:51:21.290544457 +0000 UTC m=+656.883998985" watchObservedRunningTime="2026-07-06 17:51:21.291748933 +0000 UTC m=+656.885203460"
Jul 06 17:51:21.320136 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:21.320111    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsfgb\" (UniqueName: \"kubernetes.io/projected/9b00adb8-3ac5-4dad-bc77-bd4ae0cad169-kube-api-access-dsfgb\") pod \"dns-operator-controller-manager-864dd78c4c-vdf5l\" (UID: \"9b00adb8-3ac5-4dad-bc77-bd4ae0cad169\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vdf5l"
Jul 06 17:51:21.421480 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:21.421417    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsfgb\" (UniqueName: \"kubernetes.io/projected/9b00adb8-3ac5-4dad-bc77-bd4ae0cad169-kube-api-access-dsfgb\") pod \"dns-operator-controller-manager-864dd78c4c-vdf5l\" (UID: \"9b00adb8-3ac5-4dad-bc77-bd4ae0cad169\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vdf5l"
Jul 06 17:51:21.437505 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:21.437480    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsfgb\" (UniqueName: \"kubernetes.io/projected/9b00adb8-3ac5-4dad-bc77-bd4ae0cad169-kube-api-access-dsfgb\") pod \"dns-operator-controller-manager-864dd78c4c-vdf5l\" (UID: \"9b00adb8-3ac5-4dad-bc77-bd4ae0cad169\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vdf5l"
Jul 06 17:51:21.468173 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:21.468147    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vdf5l"
Jul 06 17:51:21.587090 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:21.587063    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-vdf5l"]
Jul 06 17:51:21.588796 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:51:21.588772    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b00adb8_3ac5_4dad_bc77_bd4ae0cad169.slice/crio-eb1e93c2b4135e516dbe67531c105b0c464df5852f96bef113437a632edd3b04 WatchSource:0}: Error finding container eb1e93c2b4135e516dbe67531c105b0c464df5852f96bef113437a632edd3b04: Status 404 returned error can't find the container with id eb1e93c2b4135e516dbe67531c105b0c464df5852f96bef113437a632edd3b04
Jul 06 17:51:22.279319 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:22.279292    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vdf5l" event={"ID":"9b00adb8-3ac5-4dad-bc77-bd4ae0cad169","Type":"ContainerStarted","Data":"eb1e93c2b4135e516dbe67531c105b0c464df5852f96bef113437a632edd3b04"}
Jul 06 17:51:24.290133 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:24.290103    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vdf5l" event={"ID":"9b00adb8-3ac5-4dad-bc77-bd4ae0cad169","Type":"ContainerStarted","Data":"49032b54616edc0446601b314b188186c3647ba40e48649a24fdfb9cb0ab5f02"}
Jul 06 17:51:24.290527 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:24.290225    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vdf5l"
Jul 06 17:51:24.306872 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:24.306443    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vdf5l" podStartSLOduration=1.33349473 podStartE2EDuration="3.306426703s" totalImagesPullingTime="1.972931973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:51:24.305627549 +0000 UTC m=+659.899082080" watchObservedRunningTime="2026-07-06 17:51:24.306426703 +0000 UTC m=+659.899881233"
Jul 06 17:51:25.750605 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:25.750580    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"]
Jul 06 17:51:25.754539 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:25.754523    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:25.757068 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:25.757048    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hbc55\""
Jul 06 17:51:25.767888 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:25.767866    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"]
Jul 06 17:51:25.849914 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:25.849880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfbqp\" (UniqueName: \"kubernetes.io/projected/7a2f2544-e416-4fd0-991d-769e27720e59-kube-api-access-hfbqp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" (UID: \"7a2f2544-e416-4fd0-991d-769e27720e59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:25.850011 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:25.849924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a2f2544-e416-4fd0-991d-769e27720e59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" (UID: \"7a2f2544-e416-4fd0-991d-769e27720e59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:25.950991 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:25.950969    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfbqp\" (UniqueName: \"kubernetes.io/projected/7a2f2544-e416-4fd0-991d-769e27720e59-kube-api-access-hfbqp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" (UID: \"7a2f2544-e416-4fd0-991d-769e27720e59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:25.951076 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:25.950999    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a2f2544-e416-4fd0-991d-769e27720e59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" (UID: \"7a2f2544-e416-4fd0-991d-769e27720e59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:25.951261 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:25.951247    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a2f2544-e416-4fd0-991d-769e27720e59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" (UID: \"7a2f2544-e416-4fd0-991d-769e27720e59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:25.959450 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:25.959422    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfbqp\" (UniqueName: \"kubernetes.io/projected/7a2f2544-e416-4fd0-991d-769e27720e59-kube-api-access-hfbqp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" (UID: \"7a2f2544-e416-4fd0-991d-769e27720e59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:26.064283 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:26.064237    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:26.189868 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:26.189836    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"]
Jul 06 17:51:26.192412 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:51:26.192386    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a2f2544_e416_4fd0_991d_769e27720e59.slice/crio-7085e23c2ba60385dc90fccf2557d0143c63d7c6bfced07aca2330fd13c56582 WatchSource:0}: Error finding container 7085e23c2ba60385dc90fccf2557d0143c63d7c6bfced07aca2330fd13c56582: Status 404 returned error can't find the container with id 7085e23c2ba60385dc90fccf2557d0143c63d7c6bfced07aca2330fd13c56582
Jul 06 17:51:26.299133 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:26.299110    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw" event={"ID":"7a2f2544-e416-4fd0-991d-769e27720e59","Type":"ContainerStarted","Data":"7085e23c2ba60385dc90fccf2557d0143c63d7c6bfced07aca2330fd13c56582"}
Jul 06 17:51:29.266883 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:29.266858    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9t5th"
Jul 06 17:51:32.282052 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:32.282021    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:32.329122 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:32.329094    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw" event={"ID":"7a2f2544-e416-4fd0-991d-769e27720e59","Type":"ContainerStarted","Data":"f524877e547fbc039d038dc19b0c35b1323133eecd61e80a1f3565472bfd529a"}
Jul 06 17:51:32.329260 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:32.329188    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:32.351258 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:32.351218    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw" podStartSLOduration=1.817259332 podStartE2EDuration="7.351203565s" totalImagesPullingTime="5.533944233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:51:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:51:32.34957437 +0000 UTC m=+667.943028899" watchObservedRunningTime="2026-07-06 17:51:32.351203565 +0000 UTC m=+667.944658093"
Jul 06 17:51:35.295774 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:35.295748    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vdf5l"
Jul 06 17:51:43.336050 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:43.336019    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:45.050183 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.050152    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"]
Jul 06 17:51:45.050591 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.050315    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" containerName="manager" containerID="cri-o://f524877e547fbc039d038dc19b0c35b1323133eecd61e80a1f3565472bfd529a" gracePeriod=2
Jul 06 17:51:45.052609 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.052577    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:45.054293 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.054263    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"]
Jul 06 17:51:45.066317 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.066295    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"]
Jul 06 17:51:45.066503 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.066464    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" containerName="manager" containerID="cri-o://a38f5a319ce2ca24bbf802405d0ac410a99c5268ad133de12d4eebb84ccfd662" gracePeriod=2
Jul 06 17:51:45.072462 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.072431    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:45.081397 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.078308    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"]
Jul 06 17:51:45.081397 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.079054    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" containerName="manager"
Jul 06 17:51:45.081397 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.079073    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" containerName="manager"
Jul 06 17:51:45.081397 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.079211    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" containerName="manager"
Jul 06 17:51:45.088270 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.088248    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"]
Jul 06 17:51:45.088395 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.088376    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"
Jul 06 17:51:45.090788 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.090756    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:45.090938 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.090917    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5824s"]
Jul 06 17:51:45.091315 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.091300    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" containerName="manager"
Jul 06 17:51:45.091353 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.091320    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" containerName="manager"
Jul 06 17:51:45.091419 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.091409    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" containerName="manager"
Jul 06 17:51:45.092904 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.092866    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x2p4f\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:45.094402 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.094386    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5824s"
Jul 06 17:51:45.102768 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.102738    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"]
Jul 06 17:51:45.105727 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.105705    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5824s"]
Jul 06 17:51:45.121596 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.121566    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x2p4f\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:45.141508 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.141482    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:45.183231 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.183205    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l52nf\" (UniqueName: \"kubernetes.io/projected/c46eb0f3-b4c3-44b0-af07-9eb55481a17e-kube-api-access-l52nf\") pod \"limitador-operator-controller-manager-859d7b78bd-5824s\" (UID: \"c46eb0f3-b4c3-44b0-af07-9eb55481a17e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5824s"
Jul 06 17:51:45.183336 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.183310    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77a20eda-e785-412c-8f88-7483c28f92b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7zljp\" (UID: \"77a20eda-e785-412c-8f88-7483c28f92b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"
Jul 06 17:51:45.183408 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.183339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jchm6\" (UniqueName: \"kubernetes.io/projected/77a20eda-e785-412c-8f88-7483c28f92b5-kube-api-access-jchm6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7zljp\" (UID: \"77a20eda-e785-412c-8f88-7483c28f92b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"
Jul 06 17:51:45.283884 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.283852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l52nf\" (UniqueName: \"kubernetes.io/projected/c46eb0f3-b4c3-44b0-af07-9eb55481a17e-kube-api-access-l52nf\") pod \"limitador-operator-controller-manager-859d7b78bd-5824s\" (UID: \"c46eb0f3-b4c3-44b0-af07-9eb55481a17e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5824s"
Jul 06 17:51:45.284033 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.283937    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77a20eda-e785-412c-8f88-7483c28f92b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7zljp\" (UID: \"77a20eda-e785-412c-8f88-7483c28f92b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"
Jul 06 17:51:45.284033 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.283955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jchm6\" (UniqueName: \"kubernetes.io/projected/77a20eda-e785-412c-8f88-7483c28f92b5-kube-api-access-jchm6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7zljp\" (UID: \"77a20eda-e785-412c-8f88-7483c28f92b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"
Jul 06 17:51:45.284371 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.284344    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77a20eda-e785-412c-8f88-7483c28f92b5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7zljp\" (UID: \"77a20eda-e785-412c-8f88-7483c28f92b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"
Jul 06 17:51:45.294489 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.294375    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l52nf\" (UniqueName: \"kubernetes.io/projected/c46eb0f3-b4c3-44b0-af07-9eb55481a17e-kube-api-access-l52nf\") pod \"limitador-operator-controller-manager-859d7b78bd-5824s\" (UID: \"c46eb0f3-b4c3-44b0-af07-9eb55481a17e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5824s"
Jul 06 17:51:45.294489 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.294476    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jchm6\" (UniqueName: \"kubernetes.io/projected/77a20eda-e785-412c-8f88-7483c28f92b5-kube-api-access-jchm6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7zljp\" (UID: \"77a20eda-e785-412c-8f88-7483c28f92b5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"
Jul 06 17:51:45.303066 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.303023    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:45.305039 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.305021    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:45.305267 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.305242    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:45.307441 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.307416    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x2p4f\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:45.309327 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.309305    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:45.311111 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.311090    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x2p4f\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:45.383550 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.383518    2581 generic.go:356] "Generic (PLEG): container finished" podID="c493513b-5ab9-4c97-a647-56134ae1e36f" containerID="a38f5a319ce2ca24bbf802405d0ac410a99c5268ad133de12d4eebb84ccfd662" exitCode=0
Jul 06 17:51:45.383678 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.383559    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:45.383678 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.383564    2581 scope.go:117] "RemoveContainer" containerID="a38f5a319ce2ca24bbf802405d0ac410a99c5268ad133de12d4eebb84ccfd662"
Jul 06 17:51:45.384195 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.384177    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjd9h\" (UniqueName: \"kubernetes.io/projected/c493513b-5ab9-4c97-a647-56134ae1e36f-kube-api-access-sjd9h\") pod \"c493513b-5ab9-4c97-a647-56134ae1e36f\" (UID: \"c493513b-5ab9-4c97-a647-56134ae1e36f\") "
Jul 06 17:51:45.384278 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.384227    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a2f2544-e416-4fd0-991d-769e27720e59-extensions-socket-volume\") pod \"7a2f2544-e416-4fd0-991d-769e27720e59\" (UID: \"7a2f2544-e416-4fd0-991d-769e27720e59\") "
Jul 06 17:51:45.384278 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.384259    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfbqp\" (UniqueName: \"kubernetes.io/projected/7a2f2544-e416-4fd0-991d-769e27720e59-kube-api-access-hfbqp\") pod \"7a2f2544-e416-4fd0-991d-769e27720e59\" (UID: \"7a2f2544-e416-4fd0-991d-769e27720e59\") "
Jul 06 17:51:45.384824 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.384801    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a2f2544-e416-4fd0-991d-769e27720e59-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7a2f2544-e416-4fd0-991d-769e27720e59" (UID: "7a2f2544-e416-4fd0-991d-769e27720e59"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 17:51:45.385073 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.385050    2581 generic.go:356] "Generic (PLEG): container finished" podID="7a2f2544-e416-4fd0-991d-769e27720e59" containerID="f524877e547fbc039d038dc19b0c35b1323133eecd61e80a1f3565472bfd529a" exitCode=0
Jul 06 17:51:45.385203 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.385107    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:45.386086 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.386058    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x2p4f\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:45.386486 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.386452    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c493513b-5ab9-4c97-a647-56134ae1e36f-kube-api-access-sjd9h" (OuterVolumeSpecName: "kube-api-access-sjd9h") pod "c493513b-5ab9-4c97-a647-56134ae1e36f" (UID: "c493513b-5ab9-4c97-a647-56134ae1e36f"). InnerVolumeSpecName "kube-api-access-sjd9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:51:45.386616 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.386595    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a2f2544-e416-4fd0-991d-769e27720e59-kube-api-access-hfbqp" (OuterVolumeSpecName: "kube-api-access-hfbqp") pod "7a2f2544-e416-4fd0-991d-769e27720e59" (UID: "7a2f2544-e416-4fd0-991d-769e27720e59"). InnerVolumeSpecName "kube-api-access-hfbqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:51:45.388083 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.388061    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:45.390075 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.390042    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x2p4f\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:45.391974 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.391948    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:45.392038 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.391989    2581 scope.go:117] "RemoveContainer" containerID="a38f5a319ce2ca24bbf802405d0ac410a99c5268ad133de12d4eebb84ccfd662"
Jul 06 17:51:45.392243 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.392222    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a38f5a319ce2ca24bbf802405d0ac410a99c5268ad133de12d4eebb84ccfd662\": container with ID starting with a38f5a319ce2ca24bbf802405d0ac410a99c5268ad133de12d4eebb84ccfd662 not found: ID does not exist" containerID="a38f5a319ce2ca24bbf802405d0ac410a99c5268ad133de12d4eebb84ccfd662"
Jul 06 17:51:45.392291 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.392253    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a38f5a319ce2ca24bbf802405d0ac410a99c5268ad133de12d4eebb84ccfd662"} err="failed to get container status \"a38f5a319ce2ca24bbf802405d0ac410a99c5268ad133de12d4eebb84ccfd662\": rpc error: code = NotFound desc = could not find container \"a38f5a319ce2ca24bbf802405d0ac410a99c5268ad133de12d4eebb84ccfd662\": container with ID starting with a38f5a319ce2ca24bbf802405d0ac410a99c5268ad133de12d4eebb84ccfd662 not found: ID does not exist"
Jul 06 17:51:45.392291 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.392275    2581 scope.go:117] "RemoveContainer" containerID="f524877e547fbc039d038dc19b0c35b1323133eecd61e80a1f3565472bfd529a"
Jul 06 17:51:45.399458 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.399443    2581 scope.go:117] "RemoveContainer" containerID="f524877e547fbc039d038dc19b0c35b1323133eecd61e80a1f3565472bfd529a"
Jul 06 17:51:45.399703 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.399687    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f524877e547fbc039d038dc19b0c35b1323133eecd61e80a1f3565472bfd529a\": container with ID starting with f524877e547fbc039d038dc19b0c35b1323133eecd61e80a1f3565472bfd529a not found: ID does not exist" containerID="f524877e547fbc039d038dc19b0c35b1323133eecd61e80a1f3565472bfd529a"
Jul 06 17:51:45.399752 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.399709    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f524877e547fbc039d038dc19b0c35b1323133eecd61e80a1f3565472bfd529a"} err="failed to get container status \"f524877e547fbc039d038dc19b0c35b1323133eecd61e80a1f3565472bfd529a\": rpc error: code = NotFound desc = could not find container \"f524877e547fbc039d038dc19b0c35b1323133eecd61e80a1f3565472bfd529a\": container with ID starting with f524877e547fbc039d038dc19b0c35b1323133eecd61e80a1f3565472bfd529a not found: ID does not exist"
Jul 06 17:51:45.485066 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.485046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"
Jul 06 17:51:45.485184 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.485167    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjd9h\" (UniqueName: \"kubernetes.io/projected/c493513b-5ab9-4c97-a647-56134ae1e36f-kube-api-access-sjd9h\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:51:45.485221 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.485188    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7a2f2544-e416-4fd0-991d-769e27720e59-extensions-socket-volume\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:51:45.485221 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.485197    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfbqp\" (UniqueName: \"kubernetes.io/projected/7a2f2544-e416-4fd0-991d-769e27720e59-kube-api-access-hfbqp\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:51:45.492052 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.492037    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5824s"
Jul 06 17:51:45.621711 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.621628    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"]
Jul 06 17:51:45.626602 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:51:45.626566    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77a20eda_e785_412c_8f88_7483c28f92b5.slice/crio-b117a4f9cf5156b94d289e4370b547793c0760a211844a6a4337209c3cf81c4a WatchSource:0}: Error finding container b117a4f9cf5156b94d289e4370b547793c0760a211844a6a4337209c3cf81c4a: Status 404 returned error can't find the container with id b117a4f9cf5156b94d289e4370b547793c0760a211844a6a4337209c3cf81c4a
Jul 06 17:51:45.652863 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:45.652831    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5824s"]
Jul 06 17:51:45.663996 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:51:45.663964    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc46eb0f3_b4c3_44b0_af07_9eb55481a17e.slice/crio-ef9e86465b68bb1da2ba52f9aa2a36c0ec0336e5470d41678f87c47262e2f92f WatchSource:0}: Error finding container ef9e86465b68bb1da2ba52f9aa2a36c0ec0336e5470d41678f87c47262e2f92f: Status 404 returned error can't find the container with id ef9e86465b68bb1da2ba52f9aa2a36c0ec0336e5470d41678f87c47262e2f92f
Jul 06 17:51:45.699314 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.699286    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:45.701557 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.701537    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x2p4f\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:45.703408 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.703386    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:45.705143 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:45.705125    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x2p4f\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:46.392054 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:46.391974    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5824s" event={"ID":"c46eb0f3-b4c3-44b0-af07-9eb55481a17e","Type":"ContainerStarted","Data":"621ba2316fb9c3168fef370c232441bd37b2eaaf01cb991599989c6c4f5a883e"}
Jul 06 17:51:46.392054 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:46.392032    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5824s" event={"ID":"c46eb0f3-b4c3-44b0-af07-9eb55481a17e","Type":"ContainerStarted","Data":"ef9e86465b68bb1da2ba52f9aa2a36c0ec0336e5470d41678f87c47262e2f92f"}
Jul 06 17:51:46.392466 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:46.392091    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5824s"
Jul 06 17:51:46.393574 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:46.393553    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp" event={"ID":"77a20eda-e785-412c-8f88-7483c28f92b5","Type":"ContainerStarted","Data":"9440e798583063d2e90e2aa9a1583fe271f08325778f508d179ba75e2a15b1b6"}
Jul 06 17:51:46.393620 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:46.393596    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp" event={"ID":"77a20eda-e785-412c-8f88-7483c28f92b5","Type":"ContainerStarted","Data":"b117a4f9cf5156b94d289e4370b547793c0760a211844a6a4337209c3cf81c4a"}
Jul 06 17:51:46.393620 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:46.393617    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"
Jul 06 17:51:46.394271 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:46.394244    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x2p4f\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:46.410567 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:46.410531    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5824s" podStartSLOduration=1.410517667 podStartE2EDuration="1.410517667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:51:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:51:46.40964052 +0000 UTC m=+682.003095050" watchObservedRunningTime="2026-07-06 17:51:46.410517667 +0000 UTC m=+682.003972199"
Jul 06 17:51:46.411703 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:46.411682    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:46.413709 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:46.413691    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-75ndw\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-75ndw"
Jul 06 17:51:46.415592 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:51:46.415576    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-x2p4f\" is forbidden: User \"system:node:ip-10-0-133-236.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-236.ec2.internal' and this object" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2p4f"
Jul 06 17:51:46.433340 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:46.433305    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp" podStartSLOduration=1.433293473 podStartE2EDuration="1.433293473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:51:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:51:46.432974196 +0000 UTC m=+682.026428725" watchObservedRunningTime="2026-07-06 17:51:46.433293473 +0000 UTC m=+682.026748004"
Jul 06 17:51:46.911435 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:46.911392    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a2f2544-e416-4fd0-991d-769e27720e59" path="/var/lib/kubelet/pods/7a2f2544-e416-4fd0-991d-769e27720e59/volumes"
Jul 06 17:51:46.911781 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:46.911761    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c493513b-5ab9-4c97-a647-56134ae1e36f" path="/var/lib/kubelet/pods/c493513b-5ab9-4c97-a647-56134ae1e36f/volumes"
Jul 06 17:51:57.400852 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:57.400820    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5824s"
Jul 06 17:51:57.401333 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:51:57.400996    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"
Jul 06 17:52:14.899702 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:14.899525    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"]
Jul 06 17:52:14.900273 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:14.899812    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp" podUID="77a20eda-e785-412c-8f88-7483c28f92b5" containerName="manager" containerID="cri-o://9440e798583063d2e90e2aa9a1583fe271f08325778f508d179ba75e2a15b1b6" gracePeriod=10
Jul 06 17:52:15.093166 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.093135    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"]
Jul 06 17:52:15.098068 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.098040    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"
Jul 06 17:52:15.110688 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.110659    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"]
Jul 06 17:52:15.133234 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.133215    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"
Jul 06 17:52:15.200444 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.200416    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77a20eda-e785-412c-8f88-7483c28f92b5-extensions-socket-volume\") pod \"77a20eda-e785-412c-8f88-7483c28f92b5\" (UID: \"77a20eda-e785-412c-8f88-7483c28f92b5\") "
Jul 06 17:52:15.200587 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.200495    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jchm6\" (UniqueName: \"kubernetes.io/projected/77a20eda-e785-412c-8f88-7483c28f92b5-kube-api-access-jchm6\") pod \"77a20eda-e785-412c-8f88-7483c28f92b5\" (UID: \"77a20eda-e785-412c-8f88-7483c28f92b5\") "
Jul 06 17:52:15.200666 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.200646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhc87\" (UniqueName: \"kubernetes.io/projected/81518b18-eaf9-4c2a-8e65-aa2819d59c8d-kube-api-access-xhc87\") pod \"kuadrant-operator-controller-manager-86878c769f-4rxvl\" (UID: \"81518b18-eaf9-4c2a-8e65-aa2819d59c8d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"
Jul 06 17:52:15.200724 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.200708    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81518b18-eaf9-4c2a-8e65-aa2819d59c8d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4rxvl\" (UID: \"81518b18-eaf9-4c2a-8e65-aa2819d59c8d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"
Jul 06 17:52:15.200774 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.200753    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77a20eda-e785-412c-8f88-7483c28f92b5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "77a20eda-e785-412c-8f88-7483c28f92b5" (UID: "77a20eda-e785-412c-8f88-7483c28f92b5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 17:52:15.202532 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.202508    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77a20eda-e785-412c-8f88-7483c28f92b5-kube-api-access-jchm6" (OuterVolumeSpecName: "kube-api-access-jchm6") pod "77a20eda-e785-412c-8f88-7483c28f92b5" (UID: "77a20eda-e785-412c-8f88-7483c28f92b5"). InnerVolumeSpecName "kube-api-access-jchm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:52:15.301868 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.301831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhc87\" (UniqueName: \"kubernetes.io/projected/81518b18-eaf9-4c2a-8e65-aa2819d59c8d-kube-api-access-xhc87\") pod \"kuadrant-operator-controller-manager-86878c769f-4rxvl\" (UID: \"81518b18-eaf9-4c2a-8e65-aa2819d59c8d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"
Jul 06 17:52:15.302042 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.301877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81518b18-eaf9-4c2a-8e65-aa2819d59c8d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4rxvl\" (UID: \"81518b18-eaf9-4c2a-8e65-aa2819d59c8d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"
Jul 06 17:52:15.302042 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.301946    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/77a20eda-e785-412c-8f88-7483c28f92b5-extensions-socket-volume\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:52:15.302042 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.301956    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jchm6\" (UniqueName: \"kubernetes.io/projected/77a20eda-e785-412c-8f88-7483c28f92b5-kube-api-access-jchm6\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:52:15.302234 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.302220    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81518b18-eaf9-4c2a-8e65-aa2819d59c8d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4rxvl\" (UID: \"81518b18-eaf9-4c2a-8e65-aa2819d59c8d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"
Jul 06 17:52:15.311120 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.311101    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhc87\" (UniqueName: \"kubernetes.io/projected/81518b18-eaf9-4c2a-8e65-aa2819d59c8d-kube-api-access-xhc87\") pod \"kuadrant-operator-controller-manager-86878c769f-4rxvl\" (UID: \"81518b18-eaf9-4c2a-8e65-aa2819d59c8d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"
Jul 06 17:52:15.408869 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.408832    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"
Jul 06 17:52:15.525230 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.525195    2581 generic.go:356] "Generic (PLEG): container finished" podID="77a20eda-e785-412c-8f88-7483c28f92b5" containerID="9440e798583063d2e90e2aa9a1583fe271f08325778f508d179ba75e2a15b1b6" exitCode=0
Jul 06 17:52:15.525377 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.525239    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"
Jul 06 17:52:15.525377 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.525253    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp" event={"ID":"77a20eda-e785-412c-8f88-7483c28f92b5","Type":"ContainerDied","Data":"9440e798583063d2e90e2aa9a1583fe271f08325778f508d179ba75e2a15b1b6"}
Jul 06 17:52:15.525377 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.525307    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp" event={"ID":"77a20eda-e785-412c-8f88-7483c28f92b5","Type":"ContainerDied","Data":"b117a4f9cf5156b94d289e4370b547793c0760a211844a6a4337209c3cf81c4a"}
Jul 06 17:52:15.525377 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.525335    2581 scope.go:117] "RemoveContainer" containerID="9440e798583063d2e90e2aa9a1583fe271f08325778f508d179ba75e2a15b1b6"
Jul 06 17:52:15.533335 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.533316    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"]
Jul 06 17:52:15.535085 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.535064    2581 scope.go:117] "RemoveContainer" containerID="9440e798583063d2e90e2aa9a1583fe271f08325778f508d179ba75e2a15b1b6"
Jul 06 17:52:15.535357 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:52:15.535335    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9440e798583063d2e90e2aa9a1583fe271f08325778f508d179ba75e2a15b1b6\": container with ID starting with 9440e798583063d2e90e2aa9a1583fe271f08325778f508d179ba75e2a15b1b6 not found: ID does not exist" containerID="9440e798583063d2e90e2aa9a1583fe271f08325778f508d179ba75e2a15b1b6"
Jul 06 17:52:15.535425 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.535367    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9440e798583063d2e90e2aa9a1583fe271f08325778f508d179ba75e2a15b1b6"} err="failed to get container status \"9440e798583063d2e90e2aa9a1583fe271f08325778f508d179ba75e2a15b1b6\": rpc error: code = NotFound desc = could not find container \"9440e798583063d2e90e2aa9a1583fe271f08325778f508d179ba75e2a15b1b6\": container with ID starting with 9440e798583063d2e90e2aa9a1583fe271f08325778f508d179ba75e2a15b1b6 not found: ID does not exist"
Jul 06 17:52:15.536283 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:52:15.536260    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81518b18_eaf9_4c2a_8e65_aa2819d59c8d.slice/crio-088052f2a071573b139997823ace053eee4db734e8a1ebab3482cea08f609337 WatchSource:0}: Error finding container 088052f2a071573b139997823ace053eee4db734e8a1ebab3482cea08f609337: Status 404 returned error can't find the container with id 088052f2a071573b139997823ace053eee4db734e8a1ebab3482cea08f609337
Jul 06 17:52:15.549091 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.549074    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"]
Jul 06 17:52:15.553044 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:15.553026    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7zljp"]
Jul 06 17:52:16.531756 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:16.531721    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl" event={"ID":"81518b18-eaf9-4c2a-8e65-aa2819d59c8d","Type":"ContainerStarted","Data":"59961c48c11814c1ba3c57f49f9cb2f6709d8f04f2a79cab78bac735491659a8"}
Jul 06 17:52:16.532243 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:16.531766    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl" event={"ID":"81518b18-eaf9-4c2a-8e65-aa2819d59c8d","Type":"ContainerStarted","Data":"088052f2a071573b139997823ace053eee4db734e8a1ebab3482cea08f609337"}
Jul 06 17:52:16.532243 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:16.531797    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"
Jul 06 17:52:16.551965 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:16.551913    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl" podStartSLOduration=1.5518801500000001 podStartE2EDuration="1.55188015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:52:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:52:16.549154135 +0000 UTC m=+712.142608668" watchObservedRunningTime="2026-07-06 17:52:16.55188015 +0000 UTC m=+712.145334682"
Jul 06 17:52:16.911920 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:16.911791    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77a20eda-e785-412c-8f88-7483c28f92b5" path="/var/lib/kubelet/pods/77a20eda-e785-412c-8f88-7483c28f92b5/volumes"
Jul 06 17:52:27.538141 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:27.538109    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"
Jul 06 17:52:34.749521 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.749480    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"]
Jul 06 17:52:34.750029 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.749946    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77a20eda-e785-412c-8f88-7483c28f92b5" containerName="manager"
Jul 06 17:52:34.750029 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.749966    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="77a20eda-e785-412c-8f88-7483c28f92b5" containerName="manager"
Jul 06 17:52:34.750146 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.750060    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="77a20eda-e785-412c-8f88-7483c28f92b5" containerName="manager"
Jul 06 17:52:34.754564 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.754540    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"
Jul 06 17:52:34.757040 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.757017    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 06 17:52:34.757146 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.757079    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2ztcp\""
Jul 06 17:52:34.762604 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.762579    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"]
Jul 06 17:52:34.846564 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.846530    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/896d1315-6d43-443d-bd72-754fc6287146-config-file\") pod \"limitador-limitador-64c94fd64f-bgh7v\" (UID: \"896d1315-6d43-443d-bd72-754fc6287146\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"
Jul 06 17:52:34.846564 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.846566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzwwk\" (UniqueName: \"kubernetes.io/projected/896d1315-6d43-443d-bd72-754fc6287146-kube-api-access-dzwwk\") pod \"limitador-limitador-64c94fd64f-bgh7v\" (UID: \"896d1315-6d43-443d-bd72-754fc6287146\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"
Jul 06 17:52:34.855113 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.855078    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"]
Jul 06 17:52:34.947412 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.947376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/896d1315-6d43-443d-bd72-754fc6287146-config-file\") pod \"limitador-limitador-64c94fd64f-bgh7v\" (UID: \"896d1315-6d43-443d-bd72-754fc6287146\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"
Jul 06 17:52:34.947585 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.947435    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzwwk\" (UniqueName: \"kubernetes.io/projected/896d1315-6d43-443d-bd72-754fc6287146-kube-api-access-dzwwk\") pod \"limitador-limitador-64c94fd64f-bgh7v\" (UID: \"896d1315-6d43-443d-bd72-754fc6287146\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"
Jul 06 17:52:34.948134 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.948111    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/896d1315-6d43-443d-bd72-754fc6287146-config-file\") pod \"limitador-limitador-64c94fd64f-bgh7v\" (UID: \"896d1315-6d43-443d-bd72-754fc6287146\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"
Jul 06 17:52:34.955684 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:34.955660    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzwwk\" (UniqueName: \"kubernetes.io/projected/896d1315-6d43-443d-bd72-754fc6287146-kube-api-access-dzwwk\") pod \"limitador-limitador-64c94fd64f-bgh7v\" (UID: \"896d1315-6d43-443d-bd72-754fc6287146\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"
Jul 06 17:52:35.065506 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:35.065430    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"
Jul 06 17:52:35.191820 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:35.191798    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"]
Jul 06 17:52:35.194072 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:52:35.194045    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod896d1315_6d43_443d_bd72_754fc6287146.slice/crio-fe3c2a8202ae71e247ac242022a8336d2072e7b3837961437dea1ad3f8abbab2 WatchSource:0}: Error finding container fe3c2a8202ae71e247ac242022a8336d2072e7b3837961437dea1ad3f8abbab2: Status 404 returned error can't find the container with id fe3c2a8202ae71e247ac242022a8336d2072e7b3837961437dea1ad3f8abbab2
Jul 06 17:52:35.195820 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:35.195802    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 17:52:35.615698 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:35.615666    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v" event={"ID":"896d1315-6d43-443d-bd72-754fc6287146","Type":"ContainerStarted","Data":"fe3c2a8202ae71e247ac242022a8336d2072e7b3837961437dea1ad3f8abbab2"}
Jul 06 17:52:35.705972 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:35.705940    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-wm9dm"]
Jul 06 17:52:35.710482 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:35.710462    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wm9dm"
Jul 06 17:52:35.712818 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:35.712797    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t6bhg\""
Jul 06 17:52:35.716816 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:35.716793    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wm9dm"]
Jul 06 17:52:35.855366 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:35.855338    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88w22\" (UniqueName: \"kubernetes.io/projected/0f2ff067-5d9a-448e-928d-98832359cba0-kube-api-access-88w22\") pod \"authorino-75d556bd77-wm9dm\" (UID: \"0f2ff067-5d9a-448e-928d-98832359cba0\") " pod="kuadrant-system/authorino-75d556bd77-wm9dm"
Jul 06 17:52:35.956275 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:35.956250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88w22\" (UniqueName: \"kubernetes.io/projected/0f2ff067-5d9a-448e-928d-98832359cba0-kube-api-access-88w22\") pod \"authorino-75d556bd77-wm9dm\" (UID: \"0f2ff067-5d9a-448e-928d-98832359cba0\") " pod="kuadrant-system/authorino-75d556bd77-wm9dm"
Jul 06 17:52:35.965306 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:35.965282    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88w22\" (UniqueName: \"kubernetes.io/projected/0f2ff067-5d9a-448e-928d-98832359cba0-kube-api-access-88w22\") pod \"authorino-75d556bd77-wm9dm\" (UID: \"0f2ff067-5d9a-448e-928d-98832359cba0\") " pod="kuadrant-system/authorino-75d556bd77-wm9dm"
Jul 06 17:52:36.020857 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:36.020671    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wm9dm"
Jul 06 17:52:36.177198 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:36.177153    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wm9dm"]
Jul 06 17:52:36.180918 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:52:36.180873    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f2ff067_5d9a_448e_928d_98832359cba0.slice/crio-dd7b92b98d1f8ae69942408605ef0f899d3271733fc5aaba3d2ad7ae453a7ecd WatchSource:0}: Error finding container dd7b92b98d1f8ae69942408605ef0f899d3271733fc5aaba3d2ad7ae453a7ecd: Status 404 returned error can't find the container with id dd7b92b98d1f8ae69942408605ef0f899d3271733fc5aaba3d2ad7ae453a7ecd
Jul 06 17:52:36.622041 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:36.622004    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wm9dm" event={"ID":"0f2ff067-5d9a-448e-928d-98832359cba0","Type":"ContainerStarted","Data":"dd7b92b98d1f8ae69942408605ef0f899d3271733fc5aaba3d2ad7ae453a7ecd"}
Jul 06 17:52:40.645364 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:40.645324    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v" event={"ID":"896d1315-6d43-443d-bd72-754fc6287146","Type":"ContainerStarted","Data":"32430d8e4fbc75febe4ccb5dc0af3e24dbc5829b96ccf6bf883a9608be5787b9"}
Jul 06 17:52:40.645813 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:40.645486    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"
Jul 06 17:52:40.646738 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:40.646719    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wm9dm" event={"ID":"0f2ff067-5d9a-448e-928d-98832359cba0","Type":"ContainerStarted","Data":"babbf6504c86a73e1a3d8cc108bc7d14f854f55feacbbb2a6e6bbb03a88eed62"}
Jul 06 17:52:40.661319 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:40.661276    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v" podStartSLOduration=2.055505633 podStartE2EDuration="6.661263169s" totalImagesPullingTime="4.605757536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:52:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:52:40.660380243 +0000 UTC m=+736.253834784" watchObservedRunningTime="2026-07-06 17:52:40.661263169 +0000 UTC m=+736.254717700"
Jul 06 17:52:40.673833 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:40.673790    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-wm9dm" podStartSLOduration=2.000906683 podStartE2EDuration="5.673779206s" totalImagesPullingTime="3.672872523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:52:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:52:40.672833774 +0000 UTC m=+736.266288303" watchObservedRunningTime="2026-07-06 17:52:40.673779206 +0000 UTC m=+736.267233736"
Jul 06 17:52:51.651639 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:51.651612    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"
Jul 06 17:52:52.119966 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.119937    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"]
Jul 06 17:52:52.120159 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.120136    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v" podUID="896d1315-6d43-443d-bd72-754fc6287146" containerName="limitador" containerID="cri-o://32430d8e4fbc75febe4ccb5dc0af3e24dbc5829b96ccf6bf883a9608be5787b9" gracePeriod=30
Jul 06 17:52:52.672205 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.672181    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"
Jul 06 17:52:52.703582 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.703553    2581 generic.go:356] "Generic (PLEG): container finished" podID="896d1315-6d43-443d-bd72-754fc6287146" containerID="32430d8e4fbc75febe4ccb5dc0af3e24dbc5829b96ccf6bf883a9608be5787b9" exitCode=0
Jul 06 17:52:52.703708 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.703587    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v" event={"ID":"896d1315-6d43-443d-bd72-754fc6287146","Type":"ContainerDied","Data":"32430d8e4fbc75febe4ccb5dc0af3e24dbc5829b96ccf6bf883a9608be5787b9"}
Jul 06 17:52:52.703708 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.703633    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v" event={"ID":"896d1315-6d43-443d-bd72-754fc6287146","Type":"ContainerDied","Data":"fe3c2a8202ae71e247ac242022a8336d2072e7b3837961437dea1ad3f8abbab2"}
Jul 06 17:52:52.703708 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.703655    2581 scope.go:117] "RemoveContainer" containerID="32430d8e4fbc75febe4ccb5dc0af3e24dbc5829b96ccf6bf883a9608be5787b9"
Jul 06 17:52:52.703708 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.703592    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"
Jul 06 17:52:52.711242 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.711226    2581 scope.go:117] "RemoveContainer" containerID="32430d8e4fbc75febe4ccb5dc0af3e24dbc5829b96ccf6bf883a9608be5787b9"
Jul 06 17:52:52.711469 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:52:52.711453    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32430d8e4fbc75febe4ccb5dc0af3e24dbc5829b96ccf6bf883a9608be5787b9\": container with ID starting with 32430d8e4fbc75febe4ccb5dc0af3e24dbc5829b96ccf6bf883a9608be5787b9 not found: ID does not exist" containerID="32430d8e4fbc75febe4ccb5dc0af3e24dbc5829b96ccf6bf883a9608be5787b9"
Jul 06 17:52:52.711515 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.711477    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32430d8e4fbc75febe4ccb5dc0af3e24dbc5829b96ccf6bf883a9608be5787b9"} err="failed to get container status \"32430d8e4fbc75febe4ccb5dc0af3e24dbc5829b96ccf6bf883a9608be5787b9\": rpc error: code = NotFound desc = could not find container \"32430d8e4fbc75febe4ccb5dc0af3e24dbc5829b96ccf6bf883a9608be5787b9\": container with ID starting with 32430d8e4fbc75febe4ccb5dc0af3e24dbc5829b96ccf6bf883a9608be5787b9 not found: ID does not exist"
Jul 06 17:52:52.782983 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.782959    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/896d1315-6d43-443d-bd72-754fc6287146-config-file\") pod \"896d1315-6d43-443d-bd72-754fc6287146\" (UID: \"896d1315-6d43-443d-bd72-754fc6287146\") "
Jul 06 17:52:52.783072 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.783033    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzwwk\" (UniqueName: \"kubernetes.io/projected/896d1315-6d43-443d-bd72-754fc6287146-kube-api-access-dzwwk\") pod \"896d1315-6d43-443d-bd72-754fc6287146\" (UID: \"896d1315-6d43-443d-bd72-754fc6287146\") "
Jul 06 17:52:52.783349 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.783327    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/896d1315-6d43-443d-bd72-754fc6287146-config-file" (OuterVolumeSpecName: "config-file") pod "896d1315-6d43-443d-bd72-754fc6287146" (UID: "896d1315-6d43-443d-bd72-754fc6287146"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 17:52:52.784881 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.784858    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/896d1315-6d43-443d-bd72-754fc6287146-kube-api-access-dzwwk" (OuterVolumeSpecName: "kube-api-access-dzwwk") pod "896d1315-6d43-443d-bd72-754fc6287146" (UID: "896d1315-6d43-443d-bd72-754fc6287146"). InnerVolumeSpecName "kube-api-access-dzwwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:52:52.883928 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.883887    2581 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/896d1315-6d43-443d-bd72-754fc6287146-config-file\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:52:52.883928 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:52.883926    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzwwk\" (UniqueName: \"kubernetes.io/projected/896d1315-6d43-443d-bd72-754fc6287146-kube-api-access-dzwwk\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:52:53.021069 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:53.021015    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"]
Jul 06 17:52:53.024686 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:53.024663    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bgh7v"]
Jul 06 17:52:54.915363 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:54.915326    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="896d1315-6d43-443d-bd72-754fc6287146" path="/var/lib/kubelet/pods/896d1315-6d43-443d-bd72-754fc6287146/volumes"
Jul 06 17:52:55.549747 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.549719    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-ft2sq"]
Jul 06 17:52:55.550067 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.550052    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="896d1315-6d43-443d-bd72-754fc6287146" containerName="limitador"
Jul 06 17:52:55.550118 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.550070    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="896d1315-6d43-443d-bd72-754fc6287146" containerName="limitador"
Jul 06 17:52:55.550153 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.550128    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="896d1315-6d43-443d-bd72-754fc6287146" containerName="limitador"
Jul 06 17:52:55.553888 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.553867    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-ft2sq"
Jul 06 17:52:55.556619 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.556598    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lzhsj\""
Jul 06 17:52:55.556717 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.556598    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jul 06 17:52:55.560191 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.560170    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-ft2sq"]
Jul 06 17:52:55.701426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.701383    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cth8k\" (UniqueName: \"kubernetes.io/projected/338ff4a4-4c2b-42ef-9bd8-92e534da391d-kube-api-access-cth8k\") pod \"postgres-69fb4594cf-ft2sq\" (UID: \"338ff4a4-4c2b-42ef-9bd8-92e534da391d\") " pod="opendatahub/postgres-69fb4594cf-ft2sq"
Jul 06 17:52:55.701426 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.701421    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/338ff4a4-4c2b-42ef-9bd8-92e534da391d-data\") pod \"postgres-69fb4594cf-ft2sq\" (UID: \"338ff4a4-4c2b-42ef-9bd8-92e534da391d\") " pod="opendatahub/postgres-69fb4594cf-ft2sq"
Jul 06 17:52:55.801827 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.801767    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cth8k\" (UniqueName: \"kubernetes.io/projected/338ff4a4-4c2b-42ef-9bd8-92e534da391d-kube-api-access-cth8k\") pod \"postgres-69fb4594cf-ft2sq\" (UID: \"338ff4a4-4c2b-42ef-9bd8-92e534da391d\") " pod="opendatahub/postgres-69fb4594cf-ft2sq"
Jul 06 17:52:55.801827 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.801800    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/338ff4a4-4c2b-42ef-9bd8-92e534da391d-data\") pod \"postgres-69fb4594cf-ft2sq\" (UID: \"338ff4a4-4c2b-42ef-9bd8-92e534da391d\") " pod="opendatahub/postgres-69fb4594cf-ft2sq"
Jul 06 17:52:55.802169 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.802153    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/338ff4a4-4c2b-42ef-9bd8-92e534da391d-data\") pod \"postgres-69fb4594cf-ft2sq\" (UID: \"338ff4a4-4c2b-42ef-9bd8-92e534da391d\") " pod="opendatahub/postgres-69fb4594cf-ft2sq"
Jul 06 17:52:55.809888 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.809868    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cth8k\" (UniqueName: \"kubernetes.io/projected/338ff4a4-4c2b-42ef-9bd8-92e534da391d-kube-api-access-cth8k\") pod \"postgres-69fb4594cf-ft2sq\" (UID: \"338ff4a4-4c2b-42ef-9bd8-92e534da391d\") " pod="opendatahub/postgres-69fb4594cf-ft2sq"
Jul 06 17:52:55.866013 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.865985    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-ft2sq"
Jul 06 17:52:55.990418 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:55.990397    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-ft2sq"]
Jul 06 17:52:55.992243 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:52:55.992216    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338ff4a4_4c2b_42ef_9bd8_92e534da391d.slice/crio-557ab18d1bb2ff607967a6197ac1d32dac63b55864d871a4f88fbd55b36c72e9 WatchSource:0}: Error finding container 557ab18d1bb2ff607967a6197ac1d32dac63b55864d871a4f88fbd55b36c72e9: Status 404 returned error can't find the container with id 557ab18d1bb2ff607967a6197ac1d32dac63b55864d871a4f88fbd55b36c72e9
Jul 06 17:52:56.727227 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:52:56.727170    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-ft2sq" event={"ID":"338ff4a4-4c2b-42ef-9bd8-92e534da391d","Type":"ContainerStarted","Data":"557ab18d1bb2ff607967a6197ac1d32dac63b55864d871a4f88fbd55b36c72e9"}
Jul 06 17:53:00.748066 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:00.748030    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-ft2sq" event={"ID":"338ff4a4-4c2b-42ef-9bd8-92e534da391d","Type":"ContainerStarted","Data":"06a0b419c5e1373946b3b7affbb4d8d6dcfcfa5cf3219f778e25b40c1c94e147"}
Jul 06 17:53:00.748461 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:00.748134    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-ft2sq"
Jul 06 17:53:00.763990 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:00.763836    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-ft2sq" podStartSLOduration=1.168598738 podStartE2EDuration="5.763819317s" totalImagesPullingTime="4.595220579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:53:00.763309228 +0000 UTC m=+756.356763759" watchObservedRunningTime="2026-07-06 17:53:00.763819317 +0000 UTC m=+756.357273848"
Jul 06 17:53:06.782197 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:06.782169    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-ft2sq"
Jul 06 17:53:07.363524 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.363495    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jqfxh"]
Jul 06 17:53:07.367992 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.367975    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jqfxh"
Jul 06 17:53:07.374192 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.374170    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jqfxh"]
Jul 06 17:53:07.387592 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.387563    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5289f\" (UniqueName: \"kubernetes.io/projected/f34d95c0-9cc9-4f66-9aaf-f5411e772ef9-kube-api-access-5289f\") pod \"authorino-5d4b67c446-jqfxh\" (UID: \"f34d95c0-9cc9-4f66-9aaf-f5411e772ef9\") " pod="kuadrant-system/authorino-5d4b67c446-jqfxh"
Jul 06 17:53:07.488853 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.488829    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5289f\" (UniqueName: \"kubernetes.io/projected/f34d95c0-9cc9-4f66-9aaf-f5411e772ef9-kube-api-access-5289f\") pod \"authorino-5d4b67c446-jqfxh\" (UID: \"f34d95c0-9cc9-4f66-9aaf-f5411e772ef9\") " pod="kuadrant-system/authorino-5d4b67c446-jqfxh"
Jul 06 17:53:07.496855 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.496830    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5289f\" (UniqueName: \"kubernetes.io/projected/f34d95c0-9cc9-4f66-9aaf-f5411e772ef9-kube-api-access-5289f\") pod \"authorino-5d4b67c446-jqfxh\" (UID: \"f34d95c0-9cc9-4f66-9aaf-f5411e772ef9\") " pod="kuadrant-system/authorino-5d4b67c446-jqfxh"
Jul 06 17:53:07.663174 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.663123    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jqfxh"]
Jul 06 17:53:07.663272 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.663261    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jqfxh"
Jul 06 17:53:07.715663 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.715639    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b64b88c59-zc6xf"]
Jul 06 17:53:07.719425 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.719399    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b64b88c59-zc6xf"
Jul 06 17:53:07.727280 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.727235    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b64b88c59-zc6xf"]
Jul 06 17:53:07.788197 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.788174    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jqfxh"]
Jul 06 17:53:07.790578 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:53:07.790555    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf34d95c0_9cc9_4f66_9aaf_f5411e772ef9.slice/crio-04c265f81d9c4e662d1e77ea21f1c2d23458f1e4a292e5cdafb7a94ec9a15917 WatchSource:0}: Error finding container 04c265f81d9c4e662d1e77ea21f1c2d23458f1e4a292e5cdafb7a94ec9a15917: Status 404 returned error can't find the container with id 04c265f81d9c4e662d1e77ea21f1c2d23458f1e4a292e5cdafb7a94ec9a15917
Jul 06 17:53:07.791788 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.791764    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5wrw\" (UniqueName: \"kubernetes.io/projected/803ffbc4-d9a7-4183-ba6c-c3265eb9e718-kube-api-access-t5wrw\") pod \"authorino-b64b88c59-zc6xf\" (UID: \"803ffbc4-d9a7-4183-ba6c-c3265eb9e718\") " pod="kuadrant-system/authorino-b64b88c59-zc6xf"
Jul 06 17:53:07.865599 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.865574    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b64b88c59-zc6xf"]
Jul 06 17:53:07.865792 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:53:07.865773    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-t5wrw], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-b64b88c59-zc6xf" podUID="803ffbc4-d9a7-4183-ba6c-c3265eb9e718"
Jul 06 17:53:07.890733 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.890710    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75dcf94495-gtz4q"]
Jul 06 17:53:07.892613 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.892592    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5wrw\" (UniqueName: \"kubernetes.io/projected/803ffbc4-d9a7-4183-ba6c-c3265eb9e718-kube-api-access-t5wrw\") pod \"authorino-b64b88c59-zc6xf\" (UID: \"803ffbc4-d9a7-4183-ba6c-c3265eb9e718\") " pod="kuadrant-system/authorino-b64b88c59-zc6xf"
Jul 06 17:53:07.893764 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.893750    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75dcf94495-gtz4q"
Jul 06 17:53:07.896400 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.896382    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 06 17:53:07.902253 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.902235    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75dcf94495-gtz4q"]
Jul 06 17:53:07.904367 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.904341    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5wrw\" (UniqueName: \"kubernetes.io/projected/803ffbc4-d9a7-4183-ba6c-c3265eb9e718-kube-api-access-t5wrw\") pod \"authorino-b64b88c59-zc6xf\" (UID: \"803ffbc4-d9a7-4183-ba6c-c3265eb9e718\") " pod="kuadrant-system/authorino-b64b88c59-zc6xf"
Jul 06 17:53:07.993460 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.993440    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjjvp\" (UniqueName: \"kubernetes.io/projected/5bcc8d53-d308-4f17-be76-945bb3735c1c-kube-api-access-hjjvp\") pod \"authorino-75dcf94495-gtz4q\" (UID: \"5bcc8d53-d308-4f17-be76-945bb3735c1c\") " pod="kuadrant-system/authorino-75dcf94495-gtz4q"
Jul 06 17:53:07.993563 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:07.993469    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5bcc8d53-d308-4f17-be76-945bb3735c1c-tls-cert\") pod \"authorino-75dcf94495-gtz4q\" (UID: \"5bcc8d53-d308-4f17-be76-945bb3735c1c\") " pod="kuadrant-system/authorino-75dcf94495-gtz4q"
Jul 06 17:53:08.094020 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.093993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjjvp\" (UniqueName: \"kubernetes.io/projected/5bcc8d53-d308-4f17-be76-945bb3735c1c-kube-api-access-hjjvp\") pod \"authorino-75dcf94495-gtz4q\" (UID: \"5bcc8d53-d308-4f17-be76-945bb3735c1c\") " pod="kuadrant-system/authorino-75dcf94495-gtz4q"
Jul 06 17:53:08.094121 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.094038    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5bcc8d53-d308-4f17-be76-945bb3735c1c-tls-cert\") pod \"authorino-75dcf94495-gtz4q\" (UID: \"5bcc8d53-d308-4f17-be76-945bb3735c1c\") " pod="kuadrant-system/authorino-75dcf94495-gtz4q"
Jul 06 17:53:08.096307 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.096284    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5bcc8d53-d308-4f17-be76-945bb3735c1c-tls-cert\") pod \"authorino-75dcf94495-gtz4q\" (UID: \"5bcc8d53-d308-4f17-be76-945bb3735c1c\") " pod="kuadrant-system/authorino-75dcf94495-gtz4q"
Jul 06 17:53:08.101687 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.101666    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjjvp\" (UniqueName: \"kubernetes.io/projected/5bcc8d53-d308-4f17-be76-945bb3735c1c-kube-api-access-hjjvp\") pod \"authorino-75dcf94495-gtz4q\" (UID: \"5bcc8d53-d308-4f17-be76-945bb3735c1c\") " pod="kuadrant-system/authorino-75dcf94495-gtz4q"
Jul 06 17:53:08.202492 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.202470    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75dcf94495-gtz4q"
Jul 06 17:53:08.336080 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.336022    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75dcf94495-gtz4q"]
Jul 06 17:53:08.338791 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:53:08.338755    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5bcc8d53_d308_4f17_be76_945bb3735c1c.slice/crio-5b7c724624c4267fc6f4a863a67c84d901944e96c98cca77f5094c7286d251f2 WatchSource:0}: Error finding container 5b7c724624c4267fc6f4a863a67c84d901944e96c98cca77f5094c7286d251f2: Status 404 returned error can't find the container with id 5b7c724624c4267fc6f4a863a67c84d901944e96c98cca77f5094c7286d251f2
Jul 06 17:53:08.788852 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.788821    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75dcf94495-gtz4q" event={"ID":"5bcc8d53-d308-4f17-be76-945bb3735c1c","Type":"ContainerStarted","Data":"5b7c724624c4267fc6f4a863a67c84d901944e96c98cca77f5094c7286d251f2"}
Jul 06 17:53:08.790287 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.790258    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jqfxh" event={"ID":"f34d95c0-9cc9-4f66-9aaf-f5411e772ef9","Type":"ContainerStarted","Data":"91ed9a78bf704b202f8577cf32477eedcda4c8f53a495fd816002875ebe46a35"}
Jul 06 17:53:08.790287 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.790281    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b64b88c59-zc6xf"
Jul 06 17:53:08.790395 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.790298    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jqfxh" event={"ID":"f34d95c0-9cc9-4f66-9aaf-f5411e772ef9","Type":"ContainerStarted","Data":"04c265f81d9c4e662d1e77ea21f1c2d23458f1e4a292e5cdafb7a94ec9a15917"}
Jul 06 17:53:08.790395 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.790350    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-jqfxh" podUID="f34d95c0-9cc9-4f66-9aaf-f5411e772ef9" containerName="authorino" containerID="cri-o://91ed9a78bf704b202f8577cf32477eedcda4c8f53a495fd816002875ebe46a35" gracePeriod=30
Jul 06 17:53:08.802962 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.802945    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b64b88c59-zc6xf"
Jul 06 17:53:08.817288 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.817250    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-jqfxh" podStartSLOduration=1.371127831 podStartE2EDuration="1.817237824s" totalImagesPullingTime="446.109993ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:53:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:53:08.816279355 +0000 UTC m=+764.409733896" watchObservedRunningTime="2026-07-06 17:53:08.817237824 +0000 UTC m=+764.410692399"
Jul 06 17:53:08.900096 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.900069    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5wrw\" (UniqueName: \"kubernetes.io/projected/803ffbc4-d9a7-4183-ba6c-c3265eb9e718-kube-api-access-t5wrw\") pod \"803ffbc4-d9a7-4183-ba6c-c3265eb9e718\" (UID: \"803ffbc4-d9a7-4183-ba6c-c3265eb9e718\") "
Jul 06 17:53:08.901966 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:08.901934    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/803ffbc4-d9a7-4183-ba6c-c3265eb9e718-kube-api-access-t5wrw" (OuterVolumeSpecName: "kube-api-access-t5wrw") pod "803ffbc4-d9a7-4183-ba6c-c3265eb9e718" (UID: "803ffbc4-d9a7-4183-ba6c-c3265eb9e718"). InnerVolumeSpecName "kube-api-access-t5wrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:53:09.001295 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.001238    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5wrw\" (UniqueName: \"kubernetes.io/projected/803ffbc4-d9a7-4183-ba6c-c3265eb9e718-kube-api-access-t5wrw\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:53:09.055500 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.055476    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jqfxh"
Jul 06 17:53:09.101873 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.101852    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5289f\" (UniqueName: \"kubernetes.io/projected/f34d95c0-9cc9-4f66-9aaf-f5411e772ef9-kube-api-access-5289f\") pod \"f34d95c0-9cc9-4f66-9aaf-f5411e772ef9\" (UID: \"f34d95c0-9cc9-4f66-9aaf-f5411e772ef9\") "
Jul 06 17:53:09.103700 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.103677    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f34d95c0-9cc9-4f66-9aaf-f5411e772ef9-kube-api-access-5289f" (OuterVolumeSpecName: "kube-api-access-5289f") pod "f34d95c0-9cc9-4f66-9aaf-f5411e772ef9" (UID: "f34d95c0-9cc9-4f66-9aaf-f5411e772ef9"). InnerVolumeSpecName "kube-api-access-5289f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:53:09.203011 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.202993    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5289f\" (UniqueName: \"kubernetes.io/projected/f34d95c0-9cc9-4f66-9aaf-f5411e772ef9-kube-api-access-5289f\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:53:09.795648 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.795580    2581 generic.go:356] "Generic (PLEG): container finished" podID="f34d95c0-9cc9-4f66-9aaf-f5411e772ef9" containerID="91ed9a78bf704b202f8577cf32477eedcda4c8f53a495fd816002875ebe46a35" exitCode=0
Jul 06 17:53:09.795648 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.795626    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jqfxh"
Jul 06 17:53:09.796128 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.795668    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jqfxh" event={"ID":"f34d95c0-9cc9-4f66-9aaf-f5411e772ef9","Type":"ContainerDied","Data":"91ed9a78bf704b202f8577cf32477eedcda4c8f53a495fd816002875ebe46a35"}
Jul 06 17:53:09.796128 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.795719    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jqfxh" event={"ID":"f34d95c0-9cc9-4f66-9aaf-f5411e772ef9","Type":"ContainerDied","Data":"04c265f81d9c4e662d1e77ea21f1c2d23458f1e4a292e5cdafb7a94ec9a15917"}
Jul 06 17:53:09.796128 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.795742    2581 scope.go:117] "RemoveContainer" containerID="91ed9a78bf704b202f8577cf32477eedcda4c8f53a495fd816002875ebe46a35"
Jul 06 17:53:09.797155 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.797133    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75dcf94495-gtz4q" event={"ID":"5bcc8d53-d308-4f17-be76-945bb3735c1c","Type":"ContainerStarted","Data":"6716f2a2495bf5740063f5e24622fad820164e8b5764e1741e05ae5cb9f2e125"}
Jul 06 17:53:09.797227 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.797190    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b64b88c59-zc6xf"
Jul 06 17:53:09.804570 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.804545    2581 scope.go:117] "RemoveContainer" containerID="91ed9a78bf704b202f8577cf32477eedcda4c8f53a495fd816002875ebe46a35"
Jul 06 17:53:09.804794 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:53:09.804775    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91ed9a78bf704b202f8577cf32477eedcda4c8f53a495fd816002875ebe46a35\": container with ID starting with 91ed9a78bf704b202f8577cf32477eedcda4c8f53a495fd816002875ebe46a35 not found: ID does not exist" containerID="91ed9a78bf704b202f8577cf32477eedcda4c8f53a495fd816002875ebe46a35"
Jul 06 17:53:09.804866 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.804800    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91ed9a78bf704b202f8577cf32477eedcda4c8f53a495fd816002875ebe46a35"} err="failed to get container status \"91ed9a78bf704b202f8577cf32477eedcda4c8f53a495fd816002875ebe46a35\": rpc error: code = NotFound desc = could not find container \"91ed9a78bf704b202f8577cf32477eedcda4c8f53a495fd816002875ebe46a35\": container with ID starting with 91ed9a78bf704b202f8577cf32477eedcda4c8f53a495fd816002875ebe46a35 not found: ID does not exist"
Jul 06 17:53:09.816926 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.816857    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75dcf94495-gtz4q" podStartSLOduration=2.43743172 podStartE2EDuration="2.816843285s" totalImagesPullingTime="379.411565ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:53:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:53:09.815969073 +0000 UTC m=+765.409423622" watchObservedRunningTime="2026-07-06 17:53:09.816843285 +0000 UTC m=+765.410297816"
Jul 06 17:53:09.839649 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.839623    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b64b88c59-zc6xf"]
Jul 06 17:53:09.845818 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.845792    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wm9dm"]
Jul 06 17:53:09.846017 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.845993    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-wm9dm" podUID="0f2ff067-5d9a-448e-928d-98832359cba0" containerName="authorino" containerID="cri-o://babbf6504c86a73e1a3d8cc108bc7d14f854f55feacbbb2a6e6bbb03a88eed62" gracePeriod=30
Jul 06 17:53:09.847824 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.847797    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b64b88c59-zc6xf"]
Jul 06 17:53:09.861996 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.861975    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jqfxh"]
Jul 06 17:53:09.867711 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:09.867684    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jqfxh"]
Jul 06 17:53:10.126115 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.126091    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wm9dm"
Jul 06 17:53:10.211936 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.211881    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88w22\" (UniqueName: \"kubernetes.io/projected/0f2ff067-5d9a-448e-928d-98832359cba0-kube-api-access-88w22\") pod \"0f2ff067-5d9a-448e-928d-98832359cba0\" (UID: \"0f2ff067-5d9a-448e-928d-98832359cba0\") "
Jul 06 17:53:10.213648 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.213627    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f2ff067-5d9a-448e-928d-98832359cba0-kube-api-access-88w22" (OuterVolumeSpecName: "kube-api-access-88w22") pod "0f2ff067-5d9a-448e-928d-98832359cba0" (UID: "0f2ff067-5d9a-448e-928d-98832359cba0"). InnerVolumeSpecName "kube-api-access-88w22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:53:10.312402 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.312378    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88w22\" (UniqueName: \"kubernetes.io/projected/0f2ff067-5d9a-448e-928d-98832359cba0-kube-api-access-88w22\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:53:10.803763 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.803731    2581 generic.go:356] "Generic (PLEG): container finished" podID="0f2ff067-5d9a-448e-928d-98832359cba0" containerID="babbf6504c86a73e1a3d8cc108bc7d14f854f55feacbbb2a6e6bbb03a88eed62" exitCode=0
Jul 06 17:53:10.804173 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.803774    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wm9dm"
Jul 06 17:53:10.804173 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.803815    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wm9dm" event={"ID":"0f2ff067-5d9a-448e-928d-98832359cba0","Type":"ContainerDied","Data":"babbf6504c86a73e1a3d8cc108bc7d14f854f55feacbbb2a6e6bbb03a88eed62"}
Jul 06 17:53:10.804173 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.803852    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wm9dm" event={"ID":"0f2ff067-5d9a-448e-928d-98832359cba0","Type":"ContainerDied","Data":"dd7b92b98d1f8ae69942408605ef0f899d3271733fc5aaba3d2ad7ae453a7ecd"}
Jul 06 17:53:10.804173 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.803871    2581 scope.go:117] "RemoveContainer" containerID="babbf6504c86a73e1a3d8cc108bc7d14f854f55feacbbb2a6e6bbb03a88eed62"
Jul 06 17:53:10.812361 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.812347    2581 scope.go:117] "RemoveContainer" containerID="babbf6504c86a73e1a3d8cc108bc7d14f854f55feacbbb2a6e6bbb03a88eed62"
Jul 06 17:53:10.812602 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:53:10.812585    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"babbf6504c86a73e1a3d8cc108bc7d14f854f55feacbbb2a6e6bbb03a88eed62\": container with ID starting with babbf6504c86a73e1a3d8cc108bc7d14f854f55feacbbb2a6e6bbb03a88eed62 not found: ID does not exist" containerID="babbf6504c86a73e1a3d8cc108bc7d14f854f55feacbbb2a6e6bbb03a88eed62"
Jul 06 17:53:10.812673 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.812606    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"babbf6504c86a73e1a3d8cc108bc7d14f854f55feacbbb2a6e6bbb03a88eed62"} err="failed to get container status \"babbf6504c86a73e1a3d8cc108bc7d14f854f55feacbbb2a6e6bbb03a88eed62\": rpc error: code = NotFound desc = could not find container \"babbf6504c86a73e1a3d8cc108bc7d14f854f55feacbbb2a6e6bbb03a88eed62\": container with ID starting with babbf6504c86a73e1a3d8cc108bc7d14f854f55feacbbb2a6e6bbb03a88eed62 not found: ID does not exist"
Jul 06 17:53:10.824464 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.824444    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wm9dm"]
Jul 06 17:53:10.830156 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.830136    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wm9dm"]
Jul 06 17:53:10.911012 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.910989    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f2ff067-5d9a-448e-928d-98832359cba0" path="/var/lib/kubelet/pods/0f2ff067-5d9a-448e-928d-98832359cba0/volumes"
Jul 06 17:53:10.911295 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.911282    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="803ffbc4-d9a7-4183-ba6c-c3265eb9e718" path="/var/lib/kubelet/pods/803ffbc4-d9a7-4183-ba6c-c3265eb9e718/volumes"
Jul 06 17:53:10.911469 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:10.911458    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f34d95c0-9cc9-4f66-9aaf-f5411e772ef9" path="/var/lib/kubelet/pods/f34d95c0-9cc9-4f66-9aaf-f5411e772ef9/volumes"
Jul 06 17:53:14.375297 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.375269    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6fd8bb78-4pd49"]
Jul 06 17:53:14.375754 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.375550    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f2ff067-5d9a-448e-928d-98832359cba0" containerName="authorino"
Jul 06 17:53:14.375754 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.375560    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f2ff067-5d9a-448e-928d-98832359cba0" containerName="authorino"
Jul 06 17:53:14.375754 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.375578    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f34d95c0-9cc9-4f66-9aaf-f5411e772ef9" containerName="authorino"
Jul 06 17:53:14.375754 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.375584    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f34d95c0-9cc9-4f66-9aaf-f5411e772ef9" containerName="authorino"
Jul 06 17:53:14.375754 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.375650    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f34d95c0-9cc9-4f66-9aaf-f5411e772ef9" containerName="authorino"
Jul 06 17:53:14.375754 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.375659    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f2ff067-5d9a-448e-928d-98832359cba0" containerName="authorino"
Jul 06 17:53:14.388442 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.388420    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fd8bb78-4pd49"
Jul 06 17:53:14.390767 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.390738    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fd8bb78-4pd49"]
Jul 06 17:53:14.390980 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.390962    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 17:53:14.391081 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.391063    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 17:53:14.392305 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.392281    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cb8nb\""
Jul 06 17:53:14.438020 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.437995    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2-webhook-cert\") pod \"maas-controller-6fd8bb78-4pd49\" (UID: \"8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2\") " pod="opendatahub/maas-controller-6fd8bb78-4pd49"
Jul 06 17:53:14.438121 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.438034    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vzln\" (UniqueName: \"kubernetes.io/projected/8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2-kube-api-access-4vzln\") pod \"maas-controller-6fd8bb78-4pd49\" (UID: \"8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2\") " pod="opendatahub/maas-controller-6fd8bb78-4pd49"
Jul 06 17:53:14.539386 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.539358    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2-webhook-cert\") pod \"maas-controller-6fd8bb78-4pd49\" (UID: \"8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2\") " pod="opendatahub/maas-controller-6fd8bb78-4pd49"
Jul 06 17:53:14.539491 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.539402    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vzln\" (UniqueName: \"kubernetes.io/projected/8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2-kube-api-access-4vzln\") pod \"maas-controller-6fd8bb78-4pd49\" (UID: \"8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2\") " pod="opendatahub/maas-controller-6fd8bb78-4pd49"
Jul 06 17:53:14.541659 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.541633    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2-webhook-cert\") pod \"maas-controller-6fd8bb78-4pd49\" (UID: \"8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2\") " pod="opendatahub/maas-controller-6fd8bb78-4pd49"
Jul 06 17:53:14.549367 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.549347    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vzln\" (UniqueName: \"kubernetes.io/projected/8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2-kube-api-access-4vzln\") pod \"maas-controller-6fd8bb78-4pd49\" (UID: \"8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2\") " pod="opendatahub/maas-controller-6fd8bb78-4pd49"
Jul 06 17:53:14.554924 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.554884    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-857d875667-flnn6"]
Jul 06 17:53:14.557774 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.557758    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-857d875667-flnn6"
Jul 06 17:53:14.565265 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.565243    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-857d875667-flnn6"]
Jul 06 17:53:14.640463 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.640405    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c8bb9d8-8eee-4e08-97d6-8fd97943ed59-webhook-cert\") pod \"maas-controller-857d875667-flnn6\" (UID: \"4c8bb9d8-8eee-4e08-97d6-8fd97943ed59\") " pod="opendatahub/maas-controller-857d875667-flnn6"
Jul 06 17:53:14.640558 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.640482    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdqws\" (UniqueName: \"kubernetes.io/projected/4c8bb9d8-8eee-4e08-97d6-8fd97943ed59-kube-api-access-fdqws\") pod \"maas-controller-857d875667-flnn6\" (UID: \"4c8bb9d8-8eee-4e08-97d6-8fd97943ed59\") " pod="opendatahub/maas-controller-857d875667-flnn6"
Jul 06 17:53:14.699733 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.699713    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fd8bb78-4pd49"
Jul 06 17:53:14.741570 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.741541    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c8bb9d8-8eee-4e08-97d6-8fd97943ed59-webhook-cert\") pod \"maas-controller-857d875667-flnn6\" (UID: \"4c8bb9d8-8eee-4e08-97d6-8fd97943ed59\") " pod="opendatahub/maas-controller-857d875667-flnn6"
Jul 06 17:53:14.741678 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.741635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdqws\" (UniqueName: \"kubernetes.io/projected/4c8bb9d8-8eee-4e08-97d6-8fd97943ed59-kube-api-access-fdqws\") pod \"maas-controller-857d875667-flnn6\" (UID: \"4c8bb9d8-8eee-4e08-97d6-8fd97943ed59\") " pod="opendatahub/maas-controller-857d875667-flnn6"
Jul 06 17:53:14.744309 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.744286    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c8bb9d8-8eee-4e08-97d6-8fd97943ed59-webhook-cert\") pod \"maas-controller-857d875667-flnn6\" (UID: \"4c8bb9d8-8eee-4e08-97d6-8fd97943ed59\") " pod="opendatahub/maas-controller-857d875667-flnn6"
Jul 06 17:53:14.750274 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.750240    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdqws\" (UniqueName: \"kubernetes.io/projected/4c8bb9d8-8eee-4e08-97d6-8fd97943ed59-kube-api-access-fdqws\") pod \"maas-controller-857d875667-flnn6\" (UID: \"4c8bb9d8-8eee-4e08-97d6-8fd97943ed59\") " pod="opendatahub/maas-controller-857d875667-flnn6"
Jul 06 17:53:14.820545 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.820519    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fd8bb78-4pd49"]
Jul 06 17:53:14.822805 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:53:14.822779    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd0a0de_f8ef_4ce3_a08c_50690e2aa4b2.slice/crio-3e779b2ba45996f8c792da2bb013064ba9cbbcf4d194d697cd09fd17aa537eab WatchSource:0}: Error finding container 3e779b2ba45996f8c792da2bb013064ba9cbbcf4d194d697cd09fd17aa537eab: Status 404 returned error can't find the container with id 3e779b2ba45996f8c792da2bb013064ba9cbbcf4d194d697cd09fd17aa537eab
Jul 06 17:53:14.867351 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.867325    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-857d875667-flnn6"
Jul 06 17:53:14.994046 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:14.994022    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-857d875667-flnn6"]
Jul 06 17:53:14.995540 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:53:14.995521    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c8bb9d8_8eee_4e08_97d6_8fd97943ed59.slice/crio-90c81a4098a62fa8f803d1c260a8abd552bcb8be103c22335541909ad5e356ce WatchSource:0}: Error finding container 90c81a4098a62fa8f803d1c260a8abd552bcb8be103c22335541909ad5e356ce: Status 404 returned error can't find the container with id 90c81a4098a62fa8f803d1c260a8abd552bcb8be103c22335541909ad5e356ce
Jul 06 17:53:15.836982 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:15.836943    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fd8bb78-4pd49" event={"ID":"8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2","Type":"ContainerStarted","Data":"3e779b2ba45996f8c792da2bb013064ba9cbbcf4d194d697cd09fd17aa537eab"}
Jul 06 17:53:15.839734 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:15.839706    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-857d875667-flnn6" event={"ID":"4c8bb9d8-8eee-4e08-97d6-8fd97943ed59","Type":"ContainerStarted","Data":"90c81a4098a62fa8f803d1c260a8abd552bcb8be103c22335541909ad5e356ce"}
Jul 06 17:53:17.850099 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:17.850063    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fd8bb78-4pd49" event={"ID":"8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2","Type":"ContainerStarted","Data":"4b1638baab9bb4ca9780885353b4cf370a95dbd1da8acecc9303b989502893db"}
Jul 06 17:53:17.850565 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:17.850258    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6fd8bb78-4pd49"
Jul 06 17:53:17.851581 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:17.851559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-857d875667-flnn6" event={"ID":"4c8bb9d8-8eee-4e08-97d6-8fd97943ed59","Type":"ContainerStarted","Data":"f5af903667b3c7112dbab087c3d1caebc38f1847a0ce3eb428956e15ecacafbd"}
Jul 06 17:53:17.851690 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:17.851632    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-857d875667-flnn6"
Jul 06 17:53:17.866712 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:17.866672    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6fd8bb78-4pd49" podStartSLOduration=1.578594013 podStartE2EDuration="3.866659431s" totalImagesPullingTime="2.288065418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:53:17.865063258 +0000 UTC m=+773.458517788" watchObservedRunningTime="2026-07-06 17:53:17.866659431 +0000 UTC m=+773.460113960"
Jul 06 17:53:17.881102 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:17.881065    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-857d875667-flnn6" podStartSLOduration=1.775245097 podStartE2EDuration="3.881055726s" totalImagesPullingTime="2.105810629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:53:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:53:17.880076468 +0000 UTC m=+773.473531014" watchObservedRunningTime="2026-07-06 17:53:17.881055726 +0000 UTC m=+773.474510293"
Jul 06 17:53:28.861831 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:28.861800    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-857d875667-flnn6"
Jul 06 17:53:28.862312 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:28.862122    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6fd8bb78-4pd49"
Jul 06 17:53:28.917720 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:28.917693    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fd8bb78-4pd49"]
Jul 06 17:53:28.917881 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:28.917862    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6fd8bb78-4pd49" podUID="8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2" containerName="manager" containerID="cri-o://4b1638baab9bb4ca9780885353b4cf370a95dbd1da8acecc9303b989502893db" gracePeriod=30
Jul 06 17:53:29.164977 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.164952    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fd8bb78-4pd49"
Jul 06 17:53:29.244321 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.244294    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2-webhook-cert\") pod \"8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2\" (UID: \"8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2\") "
Jul 06 17:53:29.244450 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.244401    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vzln\" (UniqueName: \"kubernetes.io/projected/8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2-kube-api-access-4vzln\") pod \"8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2\" (UID: \"8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2\") "
Jul 06 17:53:29.246352 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.246329    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2-kube-api-access-4vzln" (OuterVolumeSpecName: "kube-api-access-4vzln") pod "8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2" (UID: "8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2"). InnerVolumeSpecName "kube-api-access-4vzln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:53:29.246469 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.246417    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2" (UID: "8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 17:53:29.344859 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.344837    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vzln\" (UniqueName: \"kubernetes.io/projected/8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2-kube-api-access-4vzln\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:53:29.344859 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.344860    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2-webhook-cert\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:53:29.904678 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.904640    2581 generic.go:356] "Generic (PLEG): container finished" podID="8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2" containerID="4b1638baab9bb4ca9780885353b4cf370a95dbd1da8acecc9303b989502893db" exitCode=0
Jul 06 17:53:29.905065 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.904686    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fd8bb78-4pd49"
Jul 06 17:53:29.905065 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.904683    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fd8bb78-4pd49" event={"ID":"8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2","Type":"ContainerDied","Data":"4b1638baab9bb4ca9780885353b4cf370a95dbd1da8acecc9303b989502893db"}
Jul 06 17:53:29.905065 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.904809    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fd8bb78-4pd49" event={"ID":"8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2","Type":"ContainerDied","Data":"3e779b2ba45996f8c792da2bb013064ba9cbbcf4d194d697cd09fd17aa537eab"}
Jul 06 17:53:29.905065 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.904839    2581 scope.go:117] "RemoveContainer" containerID="4b1638baab9bb4ca9780885353b4cf370a95dbd1da8acecc9303b989502893db"
Jul 06 17:53:29.913638 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.913621    2581 scope.go:117] "RemoveContainer" containerID="4b1638baab9bb4ca9780885353b4cf370a95dbd1da8acecc9303b989502893db"
Jul 06 17:53:29.913875 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:53:29.913859    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b1638baab9bb4ca9780885353b4cf370a95dbd1da8acecc9303b989502893db\": container with ID starting with 4b1638baab9bb4ca9780885353b4cf370a95dbd1da8acecc9303b989502893db not found: ID does not exist" containerID="4b1638baab9bb4ca9780885353b4cf370a95dbd1da8acecc9303b989502893db"
Jul 06 17:53:29.913951 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.913882    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b1638baab9bb4ca9780885353b4cf370a95dbd1da8acecc9303b989502893db"} err="failed to get container status \"4b1638baab9bb4ca9780885353b4cf370a95dbd1da8acecc9303b989502893db\": rpc error: code = NotFound desc = could not find container \"4b1638baab9bb4ca9780885353b4cf370a95dbd1da8acecc9303b989502893db\": container with ID starting with 4b1638baab9bb4ca9780885353b4cf370a95dbd1da8acecc9303b989502893db not found: ID does not exist"
Jul 06 17:53:29.924911 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.924877    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fd8bb78-4pd49"]
Jul 06 17:53:29.928244 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:29.928215    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6fd8bb78-4pd49"]
Jul 06 17:53:30.912048 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:30.912017    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2" path="/var/lib/kubelet/pods/8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2/volumes"
Jul 06 17:53:50.391528 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.391449    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5b645c74c6-wr4mv"]
Jul 06 17:53:50.391941 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.391835    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2" containerName="manager"
Jul 06 17:53:50.391941 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.391850    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2" containerName="manager"
Jul 06 17:53:50.391941 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.391929    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fd0a0de-f8ef-4ce3-a08c-50690e2aa4b2" containerName="manager"
Jul 06 17:53:50.394608 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.394593    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5b645c74c6-wr4mv"
Jul 06 17:53:50.398232 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.398207    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
Jul 06 17:53:50.398232 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.398212    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-qc69b\""
Jul 06 17:53:50.404556 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.404535    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5b645c74c6-wr4mv"]
Jul 06 17:53:50.494453 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.494421    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blhrq\" (UniqueName: \"kubernetes.io/projected/f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597-kube-api-access-blhrq\") pod \"maas-api-5b645c74c6-wr4mv\" (UID: \"f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597\") " pod="opendatahub/maas-api-5b645c74c6-wr4mv"
Jul 06 17:53:50.494637 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.494470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597-maas-api-tls\") pod \"maas-api-5b645c74c6-wr4mv\" (UID: \"f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597\") " pod="opendatahub/maas-api-5b645c74c6-wr4mv"
Jul 06 17:53:50.494637 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.494489    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597-openshift-service-ca\") pod \"maas-api-5b645c74c6-wr4mv\" (UID: \"f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597\") " pod="opendatahub/maas-api-5b645c74c6-wr4mv"
Jul 06 17:53:50.595566 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.595535    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blhrq\" (UniqueName: \"kubernetes.io/projected/f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597-kube-api-access-blhrq\") pod \"maas-api-5b645c74c6-wr4mv\" (UID: \"f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597\") " pod="opendatahub/maas-api-5b645c74c6-wr4mv"
Jul 06 17:53:50.595732 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.595589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597-maas-api-tls\") pod \"maas-api-5b645c74c6-wr4mv\" (UID: \"f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597\") " pod="opendatahub/maas-api-5b645c74c6-wr4mv"
Jul 06 17:53:50.595732 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.595615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597-openshift-service-ca\") pod \"maas-api-5b645c74c6-wr4mv\" (UID: \"f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597\") " pod="opendatahub/maas-api-5b645c74c6-wr4mv"
Jul 06 17:53:50.596343 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.596320    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597-openshift-service-ca\") pod \"maas-api-5b645c74c6-wr4mv\" (UID: \"f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597\") " pod="opendatahub/maas-api-5b645c74c6-wr4mv"
Jul 06 17:53:50.598116 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.598092    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597-maas-api-tls\") pod \"maas-api-5b645c74c6-wr4mv\" (UID: \"f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597\") " pod="opendatahub/maas-api-5b645c74c6-wr4mv"
Jul 06 17:53:50.604684 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.604661    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blhrq\" (UniqueName: \"kubernetes.io/projected/f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597-kube-api-access-blhrq\") pod \"maas-api-5b645c74c6-wr4mv\" (UID: \"f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597\") " pod="opendatahub/maas-api-5b645c74c6-wr4mv"
Jul 06 17:53:50.704742 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.704710    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5b645c74c6-wr4mv"
Jul 06 17:53:50.829043 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.829005    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-zr75s"]
Jul 06 17:53:50.832563 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.832544    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-zr75s"
Jul 06 17:53:50.835287 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.835265    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 06 17:53:50.835386 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.835315    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-w4c7f\""
Jul 06 17:53:50.844149 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.844122    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-zr75s"]
Jul 06 17:53:50.849919 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.849883    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5b645c74c6-wr4mv"]
Jul 06 17:53:50.851803 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:53:50.851779    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ae26c9_02d5_42e1_9c8a_ef9f41ce8597.slice/crio-7c35391aaba6dc86e7fdeb8fba0577eece65366a569e5e9327ebbdbe72256cf2 WatchSource:0}: Error finding container 7c35391aaba6dc86e7fdeb8fba0577eece65366a569e5e9327ebbdbe72256cf2: Status 404 returned error can't find the container with id 7c35391aaba6dc86e7fdeb8fba0577eece65366a569e5e9327ebbdbe72256cf2
Jul 06 17:53:50.877182 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.877162    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-skm4k"]
Jul 06 17:53:50.880504 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.880488    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-skm4k"
Jul 06 17:53:50.892005 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.891984    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-skm4k"]
Jul 06 17:53:50.898750 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.898730    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f15e0340-5a61-44aa-b651-331f8a5c32fe-plugins-config-volume\") pod \"payload-pre-processing-7964457696-zr75s\" (UID: \"f15e0340-5a61-44aa-b651-331f8a5c32fe\") " pod="openshift-ingress/payload-pre-processing-7964457696-zr75s"
Jul 06 17:53:50.898838 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.898764    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7jz7\" (UniqueName: \"kubernetes.io/projected/f15e0340-5a61-44aa-b651-331f8a5c32fe-kube-api-access-r7jz7\") pod \"payload-pre-processing-7964457696-zr75s\" (UID: \"f15e0340-5a61-44aa-b651-331f8a5c32fe\") " pod="openshift-ingress/payload-pre-processing-7964457696-zr75s"
Jul 06 17:53:50.999222 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.999143    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7jz7\" (UniqueName: \"kubernetes.io/projected/f15e0340-5a61-44aa-b651-331f8a5c32fe-kube-api-access-r7jz7\") pod \"payload-pre-processing-7964457696-zr75s\" (UID: \"f15e0340-5a61-44aa-b651-331f8a5c32fe\") " pod="openshift-ingress/payload-pre-processing-7964457696-zr75s"
Jul 06 17:53:50.999222 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.999192    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89d6393c-dc4c-4cf1-bfaa-b389cc6fb439-plugins-config-volume\") pod \"payload-processing-58f86c7798-skm4k\" (UID: \"89d6393c-dc4c-4cf1-bfaa-b389cc6fb439\") " pod="openshift-ingress/payload-processing-58f86c7798-skm4k"
Jul 06 17:53:50.999432 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.999297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94l67\" (UniqueName: \"kubernetes.io/projected/89d6393c-dc4c-4cf1-bfaa-b389cc6fb439-kube-api-access-94l67\") pod \"payload-processing-58f86c7798-skm4k\" (UID: \"89d6393c-dc4c-4cf1-bfaa-b389cc6fb439\") " pod="openshift-ingress/payload-processing-58f86c7798-skm4k"
Jul 06 17:53:50.999538 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:50.999518    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f15e0340-5a61-44aa-b651-331f8a5c32fe-plugins-config-volume\") pod \"payload-pre-processing-7964457696-zr75s\" (UID: \"f15e0340-5a61-44aa-b651-331f8a5c32fe\") " pod="openshift-ingress/payload-pre-processing-7964457696-zr75s"
Jul 06 17:53:51.000290 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:51.000267    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f15e0340-5a61-44aa-b651-331f8a5c32fe-plugins-config-volume\") pod \"payload-pre-processing-7964457696-zr75s\" (UID: \"f15e0340-5a61-44aa-b651-331f8a5c32fe\") " pod="openshift-ingress/payload-pre-processing-7964457696-zr75s"
Jul 06 17:53:51.001189 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:51.001169    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5b645c74c6-wr4mv" event={"ID":"f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597","Type":"ContainerStarted","Data":"7c35391aaba6dc86e7fdeb8fba0577eece65366a569e5e9327ebbdbe72256cf2"}
Jul 06 17:53:51.006834 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:51.006817    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7jz7\" (UniqueName: \"kubernetes.io/projected/f15e0340-5a61-44aa-b651-331f8a5c32fe-kube-api-access-r7jz7\") pod \"payload-pre-processing-7964457696-zr75s\" (UID: \"f15e0340-5a61-44aa-b651-331f8a5c32fe\") " pod="openshift-ingress/payload-pre-processing-7964457696-zr75s"
Jul 06 17:53:51.100882 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:51.100852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89d6393c-dc4c-4cf1-bfaa-b389cc6fb439-plugins-config-volume\") pod \"payload-processing-58f86c7798-skm4k\" (UID: \"89d6393c-dc4c-4cf1-bfaa-b389cc6fb439\") " pod="openshift-ingress/payload-processing-58f86c7798-skm4k"
Jul 06 17:53:51.101053 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:51.100925    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94l67\" (UniqueName: \"kubernetes.io/projected/89d6393c-dc4c-4cf1-bfaa-b389cc6fb439-kube-api-access-94l67\") pod \"payload-processing-58f86c7798-skm4k\" (UID: \"89d6393c-dc4c-4cf1-bfaa-b389cc6fb439\") " pod="openshift-ingress/payload-processing-58f86c7798-skm4k"
Jul 06 17:53:51.101558 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:51.101538    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89d6393c-dc4c-4cf1-bfaa-b389cc6fb439-plugins-config-volume\") pod \"payload-processing-58f86c7798-skm4k\" (UID: \"89d6393c-dc4c-4cf1-bfaa-b389cc6fb439\") " pod="openshift-ingress/payload-processing-58f86c7798-skm4k"
Jul 06 17:53:51.110125 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:51.110094    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94l67\" (UniqueName: \"kubernetes.io/projected/89d6393c-dc4c-4cf1-bfaa-b389cc6fb439-kube-api-access-94l67\") pod \"payload-processing-58f86c7798-skm4k\" (UID: \"89d6393c-dc4c-4cf1-bfaa-b389cc6fb439\") " pod="openshift-ingress/payload-processing-58f86c7798-skm4k"
Jul 06 17:53:51.146152 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:51.146128    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-zr75s"
Jul 06 17:53:51.190260 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:51.190229    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-skm4k"
Jul 06 17:53:51.287927 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:51.287877    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-zr75s"]
Jul 06 17:53:51.337722 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:51.337695    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-skm4k"]
Jul 06 17:53:51.340414 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:53:51.340377    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89d6393c_dc4c_4cf1_bfaa_b389cc6fb439.slice/crio-6075437cc789b8edad32dccd302a951a93881ff727edf4ce67424b07787781b0 WatchSource:0}: Error finding container 6075437cc789b8edad32dccd302a951a93881ff727edf4ce67424b07787781b0: Status 404 returned error can't find the container with id 6075437cc789b8edad32dccd302a951a93881ff727edf4ce67424b07787781b0
Jul 06 17:53:52.008917 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:52.008825    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-zr75s" event={"ID":"f15e0340-5a61-44aa-b651-331f8a5c32fe","Type":"ContainerStarted","Data":"81826979de36a3af7e63f6cd964d902ad4a03b07f2a12448fd7a8262b3943d30"}
Jul 06 17:53:52.010517 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:52.010485    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-skm4k" event={"ID":"89d6393c-dc4c-4cf1-bfaa-b389cc6fb439","Type":"ContainerStarted","Data":"6075437cc789b8edad32dccd302a951a93881ff727edf4ce67424b07787781b0"}
Jul 06 17:53:53.017451 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:53.017409    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-skm4k" event={"ID":"89d6393c-dc4c-4cf1-bfaa-b389cc6fb439","Type":"ContainerStarted","Data":"4906c54380be8ba357b3e95f21ae092e3c69dcc3d7ddb91d1a06ac59a02e6f61"}
Jul 06 17:53:53.017824 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:53.017477    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-skm4k"
Jul 06 17:53:53.019109 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:53.019073    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5b645c74c6-wr4mv" event={"ID":"f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597","Type":"ContainerStarted","Data":"7fc1f8687770a6807c108f90007aaa0c7cd8f8f87ecbc458a9eebb21e7463dde"}
Jul 06 17:53:53.019232 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:53.019199    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5b645c74c6-wr4mv"
Jul 06 17:53:53.020722 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:53.020699    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-zr75s" event={"ID":"f15e0340-5a61-44aa-b651-331f8a5c32fe","Type":"ContainerStarted","Data":"8aed7a12f56de55669847ef62c34572c338691a7501f77a20f8f0a6f25a9c040"}
Jul 06 17:53:53.020829 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:53.020754    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-zr75s"
Jul 06 17:53:53.032921 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:53.032868    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-skm4k" podStartSLOduration=1.4596335 podStartE2EDuration="3.032854095s" totalImagesPullingTime="1.573220595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:53:53.031305845 +0000 UTC m=+808.624760376" watchObservedRunningTime="2026-07-06 17:53:53.032854095 +0000 UTC m=+808.626308624"
Jul 06 17:53:53.045760 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:53.045722    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-zr75s" podStartSLOduration=1.429060274 podStartE2EDuration="3.045705041s" totalImagesPullingTime="1.616644767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:53:53.044707606 +0000 UTC m=+808.638162138" watchObservedRunningTime="2026-07-06 17:53:53.045705041 +0000 UTC m=+808.639159564"
Jul 06 17:53:53.062410 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:53.062362    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-5b645c74c6-wr4mv" podStartSLOduration=1.002936898 podStartE2EDuration="3.062350297s" totalImagesPullingTime="2.059413399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:53:53.060957197 +0000 UTC m=+808.654411727" watchObservedRunningTime="2026-07-06 17:53:53.062350297 +0000 UTC m=+808.655804832"
Jul 06 17:53:59.033481 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:53:59.033448    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5b645c74c6-wr4mv"
Jul 06 17:54:04.029715 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:04.029687    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-zr75s"
Jul 06 17:54:04.030223 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:04.029832    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-skm4k"
Jul 06 17:54:56.591508 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.591478    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"]
Jul 06 17:54:56.595414 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.595389    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.599365 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.599335    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 06 17:54:56.599365 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.599357    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 06 17:54:56.599567 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.599344    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 06 17:54:56.599567 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.599346    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kkmmc\""
Jul 06 17:54:56.606584 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.606564    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"]
Jul 06 17:54:56.676913 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.676860    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.676913 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.676917    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.677150 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.677059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.677150 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.677101    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.677237 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.677178    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmgxv\" (UniqueName: \"kubernetes.io/projected/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-kube-api-access-tmgxv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.677237 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.677226    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.778558 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.778523    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmgxv\" (UniqueName: \"kubernetes.io/projected/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-kube-api-access-tmgxv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.778558 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.778573    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.778803 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.778611    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.778803 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.778633    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.778803 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.778703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.779023 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.778827    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.779150 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.779097    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.779284 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.779150    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.779284 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.779198    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.780967 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.780944    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.781662 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.781643    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.785761 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.785736    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmgxv\" (UniqueName: \"kubernetes.io/projected/8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632-kube-api-access-tmgxv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w\" (UID: \"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:56.906592 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:56.906502    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:54:57.039543 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:54:57.039490    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b1c3d0f_a6f7_4ac6_900c_93ef80e7a632.slice/crio-3de64c2d59f458a411c804729ea69556a952df9c2e400cff7d76e7f2f53a3cfd WatchSource:0}: Error finding container 3de64c2d59f458a411c804729ea69556a952df9c2e400cff7d76e7f2f53a3cfd: Status 404 returned error can't find the container with id 3de64c2d59f458a411c804729ea69556a952df9c2e400cff7d76e7f2f53a3cfd
Jul 06 17:54:57.040652 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:57.040619    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"]
Jul 06 17:54:57.308015 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:57.307977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w" event={"ID":"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632","Type":"ContainerStarted","Data":"3de64c2d59f458a411c804729ea69556a952df9c2e400cff7d76e7f2f53a3cfd"}
Jul 06 17:54:58.314803 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:58.314765    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w" event={"ID":"8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632","Type":"ContainerStarted","Data":"ee2fc35e9407646319a9e8881cd6c1dda94bdc9d09dd23a2f43474542b639672"}
Jul 06 17:54:58.333031 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:54:58.332989    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w" podStartSLOduration=1.257342035 podStartE2EDuration="2.332974452s" totalImagesPullingTime="1.075632417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:54:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:54:58.331825956 +0000 UTC m=+873.925280486" watchObservedRunningTime="2026-07-06 17:54:58.332974452 +0000 UTC m=+873.926428981"
Jul 06 17:55:06.912422 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:55:06.912390    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:55:06.912864 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:55:06.912452    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:55:06.919885 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:55:06.919858    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:55:07.366783 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:55:07.366760    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w"
Jul 06 17:56:33.787525 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:33.787491    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58fbc99c78-q8tnh"]
Jul 06 17:56:33.790940 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:33.790921    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58fbc99c78-q8tnh"
Jul 06 17:56:33.797149 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:33.797128    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58fbc99c78-q8tnh"]
Jul 06 17:56:33.893625 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:33.893602    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs55n\" (UniqueName: \"kubernetes.io/projected/6d58e0a4-6519-4402-a46d-246c4068c373-kube-api-access-xs55n\") pod \"authorino-58fbc99c78-q8tnh\" (UID: \"6d58e0a4-6519-4402-a46d-246c4068c373\") " pod="kuadrant-system/authorino-58fbc99c78-q8tnh"
Jul 06 17:56:33.893733 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:33.893647    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6d58e0a4-6519-4402-a46d-246c4068c373-tls-cert\") pod \"authorino-58fbc99c78-q8tnh\" (UID: \"6d58e0a4-6519-4402-a46d-246c4068c373\") " pod="kuadrant-system/authorino-58fbc99c78-q8tnh"
Jul 06 17:56:33.994445 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:33.994421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs55n\" (UniqueName: \"kubernetes.io/projected/6d58e0a4-6519-4402-a46d-246c4068c373-kube-api-access-xs55n\") pod \"authorino-58fbc99c78-q8tnh\" (UID: \"6d58e0a4-6519-4402-a46d-246c4068c373\") " pod="kuadrant-system/authorino-58fbc99c78-q8tnh"
Jul 06 17:56:33.994556 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:33.994472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6d58e0a4-6519-4402-a46d-246c4068c373-tls-cert\") pod \"authorino-58fbc99c78-q8tnh\" (UID: \"6d58e0a4-6519-4402-a46d-246c4068c373\") " pod="kuadrant-system/authorino-58fbc99c78-q8tnh"
Jul 06 17:56:33.996888 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:33.996868    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6d58e0a4-6519-4402-a46d-246c4068c373-tls-cert\") pod \"authorino-58fbc99c78-q8tnh\" (UID: \"6d58e0a4-6519-4402-a46d-246c4068c373\") " pod="kuadrant-system/authorino-58fbc99c78-q8tnh"
Jul 06 17:56:34.002047 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:34.002029    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs55n\" (UniqueName: \"kubernetes.io/projected/6d58e0a4-6519-4402-a46d-246c4068c373-kube-api-access-xs55n\") pod \"authorino-58fbc99c78-q8tnh\" (UID: \"6d58e0a4-6519-4402-a46d-246c4068c373\") " pod="kuadrant-system/authorino-58fbc99c78-q8tnh"
Jul 06 17:56:34.100222 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:34.100168    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58fbc99c78-q8tnh"
Jul 06 17:56:34.229962 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:34.228728    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58fbc99c78-q8tnh"]
Jul 06 17:56:34.749252 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:34.749217    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58fbc99c78-q8tnh" event={"ID":"6d58e0a4-6519-4402-a46d-246c4068c373","Type":"ContainerStarted","Data":"e887f5329fcba9f67d36ed3e0f5a31c9ad32d76c8d7e4b3fc674d3397e2ec0fa"}
Jul 06 17:56:34.749387 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:34.749270    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58fbc99c78-q8tnh" event={"ID":"6d58e0a4-6519-4402-a46d-246c4068c373","Type":"ContainerStarted","Data":"ecb88ac1669217a4ab4bbde015319b2701a4aa4b53d18fb945fc384d4d3a0066"}
Jul 06 17:56:34.763886 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:34.763838    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58fbc99c78-q8tnh" podStartSLOduration=1.3324047270000001 podStartE2EDuration="1.763823319s" totalImagesPullingTime="431.418592ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:56:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:56:34.763395767 +0000 UTC m=+970.356850298" watchObservedRunningTime="2026-07-06 17:56:34.763823319 +0000 UTC m=+970.357277850"
Jul 06 17:56:34.789535 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:34.789432    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75dcf94495-gtz4q"]
Jul 06 17:56:34.790002 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:34.789720    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75dcf94495-gtz4q" podUID="5bcc8d53-d308-4f17-be76-945bb3735c1c" containerName="authorino" containerID="cri-o://6716f2a2495bf5740063f5e24622fad820164e8b5764e1741e05ae5cb9f2e125" gracePeriod=30
Jul 06 17:56:35.059254 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.059231    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75dcf94495-gtz4q"
Jul 06 17:56:35.220999 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.220915    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjjvp\" (UniqueName: \"kubernetes.io/projected/5bcc8d53-d308-4f17-be76-945bb3735c1c-kube-api-access-hjjvp\") pod \"5bcc8d53-d308-4f17-be76-945bb3735c1c\" (UID: \"5bcc8d53-d308-4f17-be76-945bb3735c1c\") "
Jul 06 17:56:35.220999 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.220994    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5bcc8d53-d308-4f17-be76-945bb3735c1c-tls-cert\") pod \"5bcc8d53-d308-4f17-be76-945bb3735c1c\" (UID: \"5bcc8d53-d308-4f17-be76-945bb3735c1c\") "
Jul 06 17:56:35.223100 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.223063    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bcc8d53-d308-4f17-be76-945bb3735c1c-kube-api-access-hjjvp" (OuterVolumeSpecName: "kube-api-access-hjjvp") pod "5bcc8d53-d308-4f17-be76-945bb3735c1c" (UID: "5bcc8d53-d308-4f17-be76-945bb3735c1c"). InnerVolumeSpecName "kube-api-access-hjjvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:56:35.231277 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.231249    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bcc8d53-d308-4f17-be76-945bb3735c1c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5bcc8d53-d308-4f17-be76-945bb3735c1c" (UID: "5bcc8d53-d308-4f17-be76-945bb3735c1c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 17:56:35.321768 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.321732    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjjvp\" (UniqueName: \"kubernetes.io/projected/5bcc8d53-d308-4f17-be76-945bb3735c1c-kube-api-access-hjjvp\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:56:35.321768 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.321762    2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5bcc8d53-d308-4f17-be76-945bb3735c1c-tls-cert\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:56:35.755350 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.755314    2581 generic.go:356] "Generic (PLEG): container finished" podID="5bcc8d53-d308-4f17-be76-945bb3735c1c" containerID="6716f2a2495bf5740063f5e24622fad820164e8b5764e1741e05ae5cb9f2e125" exitCode=0
Jul 06 17:56:35.755505 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.755364    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75dcf94495-gtz4q"
Jul 06 17:56:35.755505 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.755390    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75dcf94495-gtz4q" event={"ID":"5bcc8d53-d308-4f17-be76-945bb3735c1c","Type":"ContainerDied","Data":"6716f2a2495bf5740063f5e24622fad820164e8b5764e1741e05ae5cb9f2e125"}
Jul 06 17:56:35.755505 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.755441    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75dcf94495-gtz4q" event={"ID":"5bcc8d53-d308-4f17-be76-945bb3735c1c","Type":"ContainerDied","Data":"5b7c724624c4267fc6f4a863a67c84d901944e96c98cca77f5094c7286d251f2"}
Jul 06 17:56:35.755505 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.755459    2581 scope.go:117] "RemoveContainer" containerID="6716f2a2495bf5740063f5e24622fad820164e8b5764e1741e05ae5cb9f2e125"
Jul 06 17:56:35.765690 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.765671    2581 scope.go:117] "RemoveContainer" containerID="6716f2a2495bf5740063f5e24622fad820164e8b5764e1741e05ae5cb9f2e125"
Jul 06 17:56:35.766090 ip-10-0-133-236 kubenswrapper[2581]: E0706 17:56:35.766066    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6716f2a2495bf5740063f5e24622fad820164e8b5764e1741e05ae5cb9f2e125\": container with ID starting with 6716f2a2495bf5740063f5e24622fad820164e8b5764e1741e05ae5cb9f2e125 not found: ID does not exist" containerID="6716f2a2495bf5740063f5e24622fad820164e8b5764e1741e05ae5cb9f2e125"
Jul 06 17:56:35.766173 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.766102    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6716f2a2495bf5740063f5e24622fad820164e8b5764e1741e05ae5cb9f2e125"} err="failed to get container status \"6716f2a2495bf5740063f5e24622fad820164e8b5764e1741e05ae5cb9f2e125\": rpc error: code = NotFound desc = could not find container \"6716f2a2495bf5740063f5e24622fad820164e8b5764e1741e05ae5cb9f2e125\": container with ID starting with 6716f2a2495bf5740063f5e24622fad820164e8b5764e1741e05ae5cb9f2e125 not found: ID does not exist"
Jul 06 17:56:35.777179 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.777159    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75dcf94495-gtz4q"]
Jul 06 17:56:35.781416 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:35.781395    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75dcf94495-gtz4q"]
Jul 06 17:56:36.912072 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:56:36.912043    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bcc8d53-d308-4f17-be76-945bb3735c1c" path="/var/lib/kubelet/pods/5bcc8d53-d308-4f17-be76-945bb3735c1c/volumes"
Jul 06 17:57:52.885050 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:52.885002    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-857d875667-flnn6"]
Jul 06 17:57:52.885598 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:52.885315    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-857d875667-flnn6" podUID="4c8bb9d8-8eee-4e08-97d6-8fd97943ed59" containerName="manager" containerID="cri-o://f5af903667b3c7112dbab087c3d1caebc38f1847a0ce3eb428956e15ecacafbd" gracePeriod=30
Jul 06 17:57:53.113832 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:53.113800    2581 generic.go:356] "Generic (PLEG): container finished" podID="4c8bb9d8-8eee-4e08-97d6-8fd97943ed59" containerID="f5af903667b3c7112dbab087c3d1caebc38f1847a0ce3eb428956e15ecacafbd" exitCode=0
Jul 06 17:57:53.114044 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:53.113875    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-857d875667-flnn6" event={"ID":"4c8bb9d8-8eee-4e08-97d6-8fd97943ed59","Type":"ContainerDied","Data":"f5af903667b3c7112dbab087c3d1caebc38f1847a0ce3eb428956e15ecacafbd"}
Jul 06 17:57:53.135758 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:53.135711    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-857d875667-flnn6"
Jul 06 17:57:53.231771 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:53.231740    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdqws\" (UniqueName: \"kubernetes.io/projected/4c8bb9d8-8eee-4e08-97d6-8fd97943ed59-kube-api-access-fdqws\") pod \"4c8bb9d8-8eee-4e08-97d6-8fd97943ed59\" (UID: \"4c8bb9d8-8eee-4e08-97d6-8fd97943ed59\") "
Jul 06 17:57:53.231965 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:53.231948    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c8bb9d8-8eee-4e08-97d6-8fd97943ed59-webhook-cert\") pod \"4c8bb9d8-8eee-4e08-97d6-8fd97943ed59\" (UID: \"4c8bb9d8-8eee-4e08-97d6-8fd97943ed59\") "
Jul 06 17:57:53.233800 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:53.233763    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c8bb9d8-8eee-4e08-97d6-8fd97943ed59-kube-api-access-fdqws" (OuterVolumeSpecName: "kube-api-access-fdqws") pod "4c8bb9d8-8eee-4e08-97d6-8fd97943ed59" (UID: "4c8bb9d8-8eee-4e08-97d6-8fd97943ed59"). InnerVolumeSpecName "kube-api-access-fdqws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 17:57:53.233933 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:53.233868    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c8bb9d8-8eee-4e08-97d6-8fd97943ed59-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "4c8bb9d8-8eee-4e08-97d6-8fd97943ed59" (UID: "4c8bb9d8-8eee-4e08-97d6-8fd97943ed59"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 17:57:53.333238 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:53.333197    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c8bb9d8-8eee-4e08-97d6-8fd97943ed59-webhook-cert\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:57:53.333238 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:53.333234    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdqws\" (UniqueName: \"kubernetes.io/projected/4c8bb9d8-8eee-4e08-97d6-8fd97943ed59-kube-api-access-fdqws\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 17:57:54.119643 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.119604    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-857d875667-flnn6" event={"ID":"4c8bb9d8-8eee-4e08-97d6-8fd97943ed59","Type":"ContainerDied","Data":"90c81a4098a62fa8f803d1c260a8abd552bcb8be103c22335541909ad5e356ce"}
Jul 06 17:57:54.119643 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.119642    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-857d875667-flnn6"
Jul 06 17:57:54.120151 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.119664    2581 scope.go:117] "RemoveContainer" containerID="f5af903667b3c7112dbab087c3d1caebc38f1847a0ce3eb428956e15ecacafbd"
Jul 06 17:57:54.141020 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.140999    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-857d875667-flnn6"]
Jul 06 17:57:54.143887 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.143864    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-857d875667-flnn6"]
Jul 06 17:57:54.918110 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.918076    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c8bb9d8-8eee-4e08-97d6-8fd97943ed59" path="/var/lib/kubelet/pods/4c8bb9d8-8eee-4e08-97d6-8fd97943ed59/volumes"
Jul 06 17:57:54.935420 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.935389    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-857d875667-jtqxt"]
Jul 06 17:57:54.935739 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.935727    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c8bb9d8-8eee-4e08-97d6-8fd97943ed59" containerName="manager"
Jul 06 17:57:54.935739 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.935740    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c8bb9d8-8eee-4e08-97d6-8fd97943ed59" containerName="manager"
Jul 06 17:57:54.935849 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.935748    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bcc8d53-d308-4f17-be76-945bb3735c1c" containerName="authorino"
Jul 06 17:57:54.935849 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.935754    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bcc8d53-d308-4f17-be76-945bb3735c1c" containerName="authorino"
Jul 06 17:57:54.935849 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.935807    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bcc8d53-d308-4f17-be76-945bb3735c1c" containerName="authorino"
Jul 06 17:57:54.935849 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.935817    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c8bb9d8-8eee-4e08-97d6-8fd97943ed59" containerName="manager"
Jul 06 17:57:54.940032 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.940009    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-857d875667-jtqxt"
Jul 06 17:57:54.942485 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.942468    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 17:57:54.943755 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.943727    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cb8nb\""
Jul 06 17:57:54.943876 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.943740    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 17:57:54.949497 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:54.949473    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-857d875667-jtqxt"]
Jul 06 17:57:55.047824 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:55.047793    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ndvt\" (UniqueName: \"kubernetes.io/projected/37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3-kube-api-access-2ndvt\") pod \"maas-controller-857d875667-jtqxt\" (UID: \"37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3\") " pod="opendatahub/maas-controller-857d875667-jtqxt"
Jul 06 17:57:55.048038 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:55.047861    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3-webhook-cert\") pod \"maas-controller-857d875667-jtqxt\" (UID: \"37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3\") " pod="opendatahub/maas-controller-857d875667-jtqxt"
Jul 06 17:57:55.149246 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:55.149218    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3-webhook-cert\") pod \"maas-controller-857d875667-jtqxt\" (UID: \"37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3\") " pod="opendatahub/maas-controller-857d875667-jtqxt"
Jul 06 17:57:55.149712 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:55.149293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ndvt\" (UniqueName: \"kubernetes.io/projected/37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3-kube-api-access-2ndvt\") pod \"maas-controller-857d875667-jtqxt\" (UID: \"37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3\") " pod="opendatahub/maas-controller-857d875667-jtqxt"
Jul 06 17:57:55.151654 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:55.151623    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3-webhook-cert\") pod \"maas-controller-857d875667-jtqxt\" (UID: \"37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3\") " pod="opendatahub/maas-controller-857d875667-jtqxt"
Jul 06 17:57:55.157802 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:55.157777    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ndvt\" (UniqueName: \"kubernetes.io/projected/37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3-kube-api-access-2ndvt\") pod \"maas-controller-857d875667-jtqxt\" (UID: \"37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3\") " pod="opendatahub/maas-controller-857d875667-jtqxt"
Jul 06 17:57:55.252032 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:55.251999    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-857d875667-jtqxt"
Jul 06 17:57:55.382436 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:55.382408    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-857d875667-jtqxt"]
Jul 06 17:57:55.384136 ip-10-0-133-236 kubenswrapper[2581]: W0706 17:57:55.384112    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37cb94e1_6be3_4be5_b77f_a9ed4d6fd3e3.slice/crio-c03cbf8ca4a40f6c82ef1b87b462d39ebda89ed6d4e1adc2775b10d4d01054f2 WatchSource:0}: Error finding container c03cbf8ca4a40f6c82ef1b87b462d39ebda89ed6d4e1adc2775b10d4d01054f2: Status 404 returned error can't find the container with id c03cbf8ca4a40f6c82ef1b87b462d39ebda89ed6d4e1adc2775b10d4d01054f2
Jul 06 17:57:55.385787 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:55.385772    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 17:57:56.131112 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:56.131078    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-857d875667-jtqxt" event={"ID":"37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3","Type":"ContainerStarted","Data":"e2cf234beb521b9756e6c2855b832a7a91184f92bab16d2b1d6feeba5caf692e"}
Jul 06 17:57:56.131284 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:56.131134    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-857d875667-jtqxt" event={"ID":"37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3","Type":"ContainerStarted","Data":"c03cbf8ca4a40f6c82ef1b87b462d39ebda89ed6d4e1adc2775b10d4d01054f2"}
Jul 06 17:57:56.131284 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:56.131195    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-857d875667-jtqxt"
Jul 06 17:57:56.149387 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:57:56.149344    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-857d875667-jtqxt" podStartSLOduration=1.539731253 podStartE2EDuration="2.149330737s" totalImagesPullingTime="609.599484ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:57:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:57:56.146433354 +0000 UTC m=+1051.739887881" watchObservedRunningTime="2026-07-06 17:57:56.149330737 +0000 UTC m=+1051.742785266"
Jul 06 17:58:07.141155 ip-10-0-133-236 kubenswrapper[2581]: I0706 17:58:07.141128    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-857d875667-jtqxt"
Jul 06 18:00:00.141262 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:00.141181    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722680-n5x7t"]
Jul 06 18:00:00.144429 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:00.144412    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722680-n5x7t"
Jul 06 18:00:00.155357 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:00.155328    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722680-n5x7t"]
Jul 06 18:00:00.219590 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:00.219565    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sms4j\" (UniqueName: \"kubernetes.io/projected/1c6a6f90-ede8-4f61-b024-602381ddcf3f-kube-api-access-sms4j\") pod \"maas-api-key-cleanup-29722680-n5x7t\" (UID: \"1c6a6f90-ede8-4f61-b024-602381ddcf3f\") " pod="opendatahub/maas-api-key-cleanup-29722680-n5x7t"
Jul 06 18:00:00.320343 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:00.320313    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sms4j\" (UniqueName: \"kubernetes.io/projected/1c6a6f90-ede8-4f61-b024-602381ddcf3f-kube-api-access-sms4j\") pod \"maas-api-key-cleanup-29722680-n5x7t\" (UID: \"1c6a6f90-ede8-4f61-b024-602381ddcf3f\") " pod="opendatahub/maas-api-key-cleanup-29722680-n5x7t"
Jul 06 18:00:00.328487 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:00.328463    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sms4j\" (UniqueName: \"kubernetes.io/projected/1c6a6f90-ede8-4f61-b024-602381ddcf3f-kube-api-access-sms4j\") pod \"maas-api-key-cleanup-29722680-n5x7t\" (UID: \"1c6a6f90-ede8-4f61-b024-602381ddcf3f\") " pod="opendatahub/maas-api-key-cleanup-29722680-n5x7t"
Jul 06 18:00:00.454967 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:00.454946    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722680-n5x7t"
Jul 06 18:00:00.584800 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:00.584773    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722680-n5x7t"]
Jul 06 18:00:00.587303 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:00:00.587279    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6a6f90_ede8_4f61_b024_602381ddcf3f.slice/crio-db5f5f496e3e7be3a0288b9cde451b0a09a53f1349f59275f480893ca15e6a08 WatchSource:0}: Error finding container db5f5f496e3e7be3a0288b9cde451b0a09a53f1349f59275f480893ca15e6a08: Status 404 returned error can't find the container with id db5f5f496e3e7be3a0288b9cde451b0a09a53f1349f59275f480893ca15e6a08
Jul 06 18:00:00.711181 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:00.711121    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722680-n5x7t" event={"ID":"1c6a6f90-ede8-4f61-b024-602381ddcf3f","Type":"ContainerStarted","Data":"db5f5f496e3e7be3a0288b9cde451b0a09a53f1349f59275f480893ca15e6a08"}
Jul 06 18:00:20.808876 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:20.808791    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722680-n5x7t" event={"ID":"1c6a6f90-ede8-4f61-b024-602381ddcf3f","Type":"ContainerStarted","Data":"25b043364ef0a975a78d33f408185c971fa70a3229cca848546b1d842f5981a8"}
Jul 06 18:00:20.826022 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:20.825975    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722680-n5x7t" podStartSLOduration=1.244506461 podStartE2EDuration="20.825960729s" totalImagesPullingTime="19.581454268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:00:20.824576358 +0000 UTC m=+1196.418030888" watchObservedRunningTime="2026-07-06 18:00:20.825960729 +0000 UTC m=+1196.419415315"
Jul 06 18:00:21.814826 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:21.814795    2581 generic.go:356] "Generic (PLEG): container finished" podID="1c6a6f90-ede8-4f61-b024-602381ddcf3f" containerID="25b043364ef0a975a78d33f408185c971fa70a3229cca848546b1d842f5981a8" exitCode=0
Jul 06 18:00:21.815219 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:21.814863    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722680-n5x7t" event={"ID":"1c6a6f90-ede8-4f61-b024-602381ddcf3f","Type":"ContainerDied","Data":"25b043364ef0a975a78d33f408185c971fa70a3229cca848546b1d842f5981a8"}
Jul 06 18:00:22.960270 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:22.960243    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722680-n5x7t"
Jul 06 18:00:23.106339 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:23.106251    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sms4j\" (UniqueName: \"kubernetes.io/projected/1c6a6f90-ede8-4f61-b024-602381ddcf3f-kube-api-access-sms4j\") pod \"1c6a6f90-ede8-4f61-b024-602381ddcf3f\" (UID: \"1c6a6f90-ede8-4f61-b024-602381ddcf3f\") "
Jul 06 18:00:23.108244 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:23.108213    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c6a6f90-ede8-4f61-b024-602381ddcf3f-kube-api-access-sms4j" (OuterVolumeSpecName: "kube-api-access-sms4j") pod "1c6a6f90-ede8-4f61-b024-602381ddcf3f" (UID: "1c6a6f90-ede8-4f61-b024-602381ddcf3f"). InnerVolumeSpecName "kube-api-access-sms4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:00:23.207011 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:23.206980    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sms4j\" (UniqueName: \"kubernetes.io/projected/1c6a6f90-ede8-4f61-b024-602381ddcf3f-kube-api-access-sms4j\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:00:23.825635 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:23.825602    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722680-n5x7t"
Jul 06 18:00:23.825821 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:23.825640    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722680-n5x7t" event={"ID":"1c6a6f90-ede8-4f61-b024-602381ddcf3f","Type":"ContainerDied","Data":"db5f5f496e3e7be3a0288b9cde451b0a09a53f1349f59275f480893ca15e6a08"}
Jul 06 18:00:23.825821 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:00:23.825697    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db5f5f496e3e7be3a0288b9cde451b0a09a53f1349f59275f480893ca15e6a08"
Jul 06 18:02:39.302681 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:39.302636    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"]
Jul 06 18:02:39.303222 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:39.302858    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl" podUID="81518b18-eaf9-4c2a-8e65-aa2819d59c8d" containerName="manager" containerID="cri-o://59961c48c11814c1ba3c57f49f9cb2f6709d8f04f2a79cab78bac735491659a8" gracePeriod=10
Jul 06 18:02:42.162013 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.161988    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"
Jul 06 18:02:42.292743 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.292648    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhc87\" (UniqueName: \"kubernetes.io/projected/81518b18-eaf9-4c2a-8e65-aa2819d59c8d-kube-api-access-xhc87\") pod \"81518b18-eaf9-4c2a-8e65-aa2819d59c8d\" (UID: \"81518b18-eaf9-4c2a-8e65-aa2819d59c8d\") "
Jul 06 18:02:42.292743 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.292688    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81518b18-eaf9-4c2a-8e65-aa2819d59c8d-extensions-socket-volume\") pod \"81518b18-eaf9-4c2a-8e65-aa2819d59c8d\" (UID: \"81518b18-eaf9-4c2a-8e65-aa2819d59c8d\") "
Jul 06 18:02:42.293104 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.293077    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81518b18-eaf9-4c2a-8e65-aa2819d59c8d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "81518b18-eaf9-4c2a-8e65-aa2819d59c8d" (UID: "81518b18-eaf9-4c2a-8e65-aa2819d59c8d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:02:42.294727 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.294704    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81518b18-eaf9-4c2a-8e65-aa2819d59c8d-kube-api-access-xhc87" (OuterVolumeSpecName: "kube-api-access-xhc87") pod "81518b18-eaf9-4c2a-8e65-aa2819d59c8d" (UID: "81518b18-eaf9-4c2a-8e65-aa2819d59c8d"). InnerVolumeSpecName "kube-api-access-xhc87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:02:42.394170 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.394141    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhc87\" (UniqueName: \"kubernetes.io/projected/81518b18-eaf9-4c2a-8e65-aa2819d59c8d-kube-api-access-xhc87\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:02:42.394170 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.394167    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81518b18-eaf9-4c2a-8e65-aa2819d59c8d-extensions-socket-volume\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:02:42.472672 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.472644    2581 generic.go:356] "Generic (PLEG): container finished" podID="81518b18-eaf9-4c2a-8e65-aa2819d59c8d" containerID="59961c48c11814c1ba3c57f49f9cb2f6709d8f04f2a79cab78bac735491659a8" exitCode=0
Jul 06 18:02:42.472780 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.472682    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl" event={"ID":"81518b18-eaf9-4c2a-8e65-aa2819d59c8d","Type":"ContainerDied","Data":"59961c48c11814c1ba3c57f49f9cb2f6709d8f04f2a79cab78bac735491659a8"}
Jul 06 18:02:42.472780 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.472697    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"
Jul 06 18:02:42.472780 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.472736    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl" event={"ID":"81518b18-eaf9-4c2a-8e65-aa2819d59c8d","Type":"ContainerDied","Data":"088052f2a071573b139997823ace053eee4db734e8a1ebab3482cea08f609337"}
Jul 06 18:02:42.472780 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.472767    2581 scope.go:117] "RemoveContainer" containerID="59961c48c11814c1ba3c57f49f9cb2f6709d8f04f2a79cab78bac735491659a8"
Jul 06 18:02:42.482694 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.482679    2581 scope.go:117] "RemoveContainer" containerID="59961c48c11814c1ba3c57f49f9cb2f6709d8f04f2a79cab78bac735491659a8"
Jul 06 18:02:42.482938 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:02:42.482917    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59961c48c11814c1ba3c57f49f9cb2f6709d8f04f2a79cab78bac735491659a8\": container with ID starting with 59961c48c11814c1ba3c57f49f9cb2f6709d8f04f2a79cab78bac735491659a8 not found: ID does not exist" containerID="59961c48c11814c1ba3c57f49f9cb2f6709d8f04f2a79cab78bac735491659a8"
Jul 06 18:02:42.482991 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.482944    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59961c48c11814c1ba3c57f49f9cb2f6709d8f04f2a79cab78bac735491659a8"} err="failed to get container status \"59961c48c11814c1ba3c57f49f9cb2f6709d8f04f2a79cab78bac735491659a8\": rpc error: code = NotFound desc = could not find container \"59961c48c11814c1ba3c57f49f9cb2f6709d8f04f2a79cab78bac735491659a8\": container with ID starting with 59961c48c11814c1ba3c57f49f9cb2f6709d8f04f2a79cab78bac735491659a8 not found: ID does not exist"
Jul 06 18:02:42.493746 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.493720    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"]
Jul 06 18:02:42.497864 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.497838    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4rxvl"]
Jul 06 18:02:42.911997 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:02:42.911976    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81518b18-eaf9-4c2a-8e65-aa2819d59c8d" path="/var/lib/kubelet/pods/81518b18-eaf9-4c2a-8e65-aa2819d59c8d/volumes"
Jul 06 18:03:40.427492 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.427461    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6"]
Jul 06 18:03:40.427865 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.427771    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81518b18-eaf9-4c2a-8e65-aa2819d59c8d" containerName="manager"
Jul 06 18:03:40.427865 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.427781    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="81518b18-eaf9-4c2a-8e65-aa2819d59c8d" containerName="manager"
Jul 06 18:03:40.427865 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.427794    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c6a6f90-ede8-4f61-b024-602381ddcf3f" containerName="cleanup"
Jul 06 18:03:40.427865 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.427800    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c6a6f90-ede8-4f61-b024-602381ddcf3f" containerName="cleanup"
Jul 06 18:03:40.428015 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.427877    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="81518b18-eaf9-4c2a-8e65-aa2819d59c8d" containerName="manager"
Jul 06 18:03:40.428015 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.427884    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c6a6f90-ede8-4f61-b024-602381ddcf3f" containerName="cleanup"
Jul 06 18:03:40.431782 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.431765    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6"
Jul 06 18:03:40.434331 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.434317    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hbc55\""
Jul 06 18:03:40.440545 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.440524    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6"]
Jul 06 18:03:40.497973 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.497945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41a5db61-c70f-4b34-8e12-2200729b3d36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qzgh6\" (UID: \"41a5db61-c70f-4b34-8e12-2200729b3d36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6"
Jul 06 18:03:40.498094 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.498007    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpt77\" (UniqueName: \"kubernetes.io/projected/41a5db61-c70f-4b34-8e12-2200729b3d36-kube-api-access-jpt77\") pod \"kuadrant-operator-controller-manager-86878c769f-qzgh6\" (UID: \"41a5db61-c70f-4b34-8e12-2200729b3d36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6"
Jul 06 18:03:40.598460 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.598433    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpt77\" (UniqueName: \"kubernetes.io/projected/41a5db61-c70f-4b34-8e12-2200729b3d36-kube-api-access-jpt77\") pod \"kuadrant-operator-controller-manager-86878c769f-qzgh6\" (UID: \"41a5db61-c70f-4b34-8e12-2200729b3d36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6"
Jul 06 18:03:40.598579 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.598484    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41a5db61-c70f-4b34-8e12-2200729b3d36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qzgh6\" (UID: \"41a5db61-c70f-4b34-8e12-2200729b3d36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6"
Jul 06 18:03:40.598765 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.598747    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41a5db61-c70f-4b34-8e12-2200729b3d36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qzgh6\" (UID: \"41a5db61-c70f-4b34-8e12-2200729b3d36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6"
Jul 06 18:03:40.611627 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.611597    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpt77\" (UniqueName: \"kubernetes.io/projected/41a5db61-c70f-4b34-8e12-2200729b3d36-kube-api-access-jpt77\") pod \"kuadrant-operator-controller-manager-86878c769f-qzgh6\" (UID: \"41a5db61-c70f-4b34-8e12-2200729b3d36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6"
Jul 06 18:03:40.743406 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.743381    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6"
Jul 06 18:03:40.873752 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:40.873723    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6"]
Jul 06 18:03:40.874966 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:03:40.874935    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a5db61_c70f_4b34_8e12_2200729b3d36.slice/crio-a7162fef9a50a09750f6bac13836723a885513e155ada3ea69640877e4811bc3 WatchSource:0}: Error finding container a7162fef9a50a09750f6bac13836723a885513e155ada3ea69640877e4811bc3: Status 404 returned error can't find the container with id a7162fef9a50a09750f6bac13836723a885513e155ada3ea69640877e4811bc3
Jul 06 18:03:41.739485 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:41.739446    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6" event={"ID":"41a5db61-c70f-4b34-8e12-2200729b3d36","Type":"ContainerStarted","Data":"604b276f825c180dfc20d169bc715c3e781904a97455131952f0332117596456"}
Jul 06 18:03:41.739843 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:41.739511    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6" event={"ID":"41a5db61-c70f-4b34-8e12-2200729b3d36","Type":"ContainerStarted","Data":"a7162fef9a50a09750f6bac13836723a885513e155ada3ea69640877e4811bc3"}
Jul 06 18:03:41.739843 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:41.739549    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6"
Jul 06 18:03:41.771265 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:41.771208    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6" podStartSLOduration=1.771190389 podStartE2EDuration="1.771190389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:03:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:03:41.769689037 +0000 UTC m=+1397.363143568" watchObservedRunningTime="2026-07-06 18:03:41.771190389 +0000 UTC m=+1397.364644920"
Jul 06 18:03:52.746065 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:03:52.746037    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzgh6"
Jul 06 18:10:13.027451 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.027408    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"]
Jul 06 18:10:13.031624 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.031592    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"
Jul 06 18:10:13.034710 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.034677    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-12b475ae-serving-cert\""
Jul 06 18:10:13.044853 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.044812    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"]
Jul 06 18:10:13.137190 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.137147    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-maas-api-tls\") pod \"maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw\" (UID: \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\") " pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"
Jul 06 18:10:13.137384 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.137214    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7d2n\" (UniqueName: \"kubernetes.io/projected/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-kube-api-access-b7d2n\") pod \"maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw\" (UID: \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\") " pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"
Jul 06 18:10:13.137384 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.137269    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-openshift-service-ca\") pod \"maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw\" (UID: \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\") " pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"
Jul 06 18:10:13.238593 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.238550    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-maas-api-tls\") pod \"maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw\" (UID: \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\") " pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"
Jul 06 18:10:13.238785 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.238645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7d2n\" (UniqueName: \"kubernetes.io/projected/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-kube-api-access-b7d2n\") pod \"maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw\" (UID: \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\") " pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"
Jul 06 18:10:13.238785 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.238682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-openshift-service-ca\") pod \"maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw\" (UID: \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\") " pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"
Jul 06 18:10:13.239577 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.239546    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-openshift-service-ca\") pod \"maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw\" (UID: \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\") " pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"
Jul 06 18:10:13.241588 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.241557    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-maas-api-tls\") pod \"maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw\" (UID: \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\") " pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"
Jul 06 18:10:13.248168 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.248138    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7d2n\" (UniqueName: \"kubernetes.io/projected/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-kube-api-access-b7d2n\") pod \"maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw\" (UID: \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\") " pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"
Jul 06 18:10:13.346141 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.346038    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"
Jul 06 18:10:13.501198 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.501169    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"]
Jul 06 18:10:13.503480 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:10:13.503441    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ba91a03_ca70_43c1_a833_f9ecbb7f316d.slice/crio-0819a6d6300932136d4fe95f344976d370f3aa46ea429fab4cb5ef8acada4843 WatchSource:0}: Error finding container 0819a6d6300932136d4fe95f344976d370f3aa46ea429fab4cb5ef8acada4843: Status 404 returned error can't find the container with id 0819a6d6300932136d4fe95f344976d370f3aa46ea429fab4cb5ef8acada4843
Jul 06 18:10:13.504938 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.504886    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 18:10:13.648773 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:13.648674    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw" event={"ID":"9ba91a03-ca70-43c1-a833-f9ecbb7f316d","Type":"ContainerStarted","Data":"0819a6d6300932136d4fe95f344976d370f3aa46ea429fab4cb5ef8acada4843"}
Jul 06 18:10:14.656126 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:14.656031    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw" event={"ID":"9ba91a03-ca70-43c1-a833-f9ecbb7f316d","Type":"ContainerStarted","Data":"854a9b60aa3929b186c1021b49da13d5c3cc95baea2939b56feb49567eef6e6a"}
Jul 06 18:10:14.656496 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:14.656147    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"
Jul 06 18:10:14.673017 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:14.672966    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw" podStartSLOduration=0.960843676 podStartE2EDuration="1.672949732s" totalImagesPullingTime="712.106056ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:10:14.671325192 +0000 UTC m=+1790.264779720" watchObservedRunningTime="2026-07-06 18:10:14.672949732 +0000 UTC m=+1790.266404261"
Jul 06 18:10:18.486575 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:18.486535    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"]
Jul 06 18:10:18.487205 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:18.486748    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw" podUID="9ba91a03-ca70-43c1-a833-f9ecbb7f316d" containerName="maas-api" containerID="cri-o://854a9b60aa3929b186c1021b49da13d5c3cc95baea2939b56feb49567eef6e6a" gracePeriod=30
Jul 06 18:10:18.678316 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:18.678260    2581 generic.go:356] "Generic (PLEG): container finished" podID="9ba91a03-ca70-43c1-a833-f9ecbb7f316d" containerID="854a9b60aa3929b186c1021b49da13d5c3cc95baea2939b56feb49567eef6e6a" exitCode=0
Jul 06 18:10:18.678517 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:18.678340    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw" event={"ID":"9ba91a03-ca70-43c1-a833-f9ecbb7f316d","Type":"ContainerDied","Data":"854a9b60aa3929b186c1021b49da13d5c3cc95baea2939b56feb49567eef6e6a"}
Jul 06 18:10:18.752870 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:18.752790    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"
Jul 06 18:10:18.885531 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:18.885484    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7d2n\" (UniqueName: \"kubernetes.io/projected/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-kube-api-access-b7d2n\") pod \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\" (UID: \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\") "
Jul 06 18:10:18.885742 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:18.885594    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-openshift-service-ca\") pod \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\" (UID: \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\") "
Jul 06 18:10:18.885742 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:18.885644    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-maas-api-tls\") pod \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\" (UID: \"9ba91a03-ca70-43c1-a833-f9ecbb7f316d\") "
Jul 06 18:10:18.886017 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:18.885974    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ba91a03-ca70-43c1-a833-f9ecbb7f316d" (UID: "9ba91a03-ca70-43c1-a833-f9ecbb7f316d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:10:18.887796 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:18.887767    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ba91a03-ca70-43c1-a833-f9ecbb7f316d" (UID: "9ba91a03-ca70-43c1-a833-f9ecbb7f316d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:10:18.887942 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:18.887791    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-kube-api-access-b7d2n" (OuterVolumeSpecName: "kube-api-access-b7d2n") pod "9ba91a03-ca70-43c1-a833-f9ecbb7f316d" (UID: "9ba91a03-ca70-43c1-a833-f9ecbb7f316d"). InnerVolumeSpecName "kube-api-access-b7d2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:10:18.986538 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:18.986499    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-openshift-service-ca\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:10:18.986538 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:18.986535    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-maas-api-tls\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:10:18.986538 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:18.986549    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7d2n\" (UniqueName: \"kubernetes.io/projected/9ba91a03-ca70-43c1-a833-f9ecbb7f316d-kube-api-access-b7d2n\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:10:19.685088 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:19.685044    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw" event={"ID":"9ba91a03-ca70-43c1-a833-f9ecbb7f316d","Type":"ContainerDied","Data":"0819a6d6300932136d4fe95f344976d370f3aa46ea429fab4cb5ef8acada4843"}
Jul 06 18:10:19.685088 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:19.685080    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"
Jul 06 18:10:19.685610 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:19.685129    2581 scope.go:117] "RemoveContainer" containerID="854a9b60aa3929b186c1021b49da13d5c3cc95baea2939b56feb49567eef6e6a"
Jul 06 18:10:19.701623 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:19.701574    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"]
Jul 06 18:10:19.703583 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:19.703553    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-12b475ae-5c88f5c764-7wtkw"]
Jul 06 18:10:20.913493 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:20.913455    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ba91a03-ca70-43c1-a833-f9ecbb7f316d" path="/var/lib/kubelet/pods/9ba91a03-ca70-43c1-a833-f9ecbb7f316d/volumes"
Jul 06 18:10:25.661699 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.661661    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"]
Jul 06 18:10:25.662159 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.662141    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ba91a03-ca70-43c1-a833-f9ecbb7f316d" containerName="maas-api"
Jul 06 18:10:25.662203 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.662163    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ba91a03-ca70-43c1-a833-f9ecbb7f316d" containerName="maas-api"
Jul 06 18:10:25.662324 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.662310    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ba91a03-ca70-43c1-a833-f9ecbb7f316d" containerName="maas-api"
Jul 06 18:10:25.667018 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.666984    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:25.669484 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.669460    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-5b603d23-serving-cert\""
Jul 06 18:10:25.675756 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.675336    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"]
Jul 06 18:10:25.749179 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.749140    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xz8z\" (UniqueName: \"kubernetes.io/projected/7614f7af-4710-4724-a15a-9b15ad4ea0d1-kube-api-access-7xz8z\") pod \"maas-api-e2e-ait-5b603d23-7f474cb858-qbpts\" (UID: \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\") " pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:25.749372 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.749233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7614f7af-4710-4724-a15a-9b15ad4ea0d1-maas-api-tls\") pod \"maas-api-e2e-ait-5b603d23-7f474cb858-qbpts\" (UID: \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\") " pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:25.749372 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.749286    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7614f7af-4710-4724-a15a-9b15ad4ea0d1-openshift-service-ca\") pod \"maas-api-e2e-ait-5b603d23-7f474cb858-qbpts\" (UID: \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\") " pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:25.850167 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.850119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xz8z\" (UniqueName: \"kubernetes.io/projected/7614f7af-4710-4724-a15a-9b15ad4ea0d1-kube-api-access-7xz8z\") pod \"maas-api-e2e-ait-5b603d23-7f474cb858-qbpts\" (UID: \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\") " pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:25.850376 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.850207    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7614f7af-4710-4724-a15a-9b15ad4ea0d1-maas-api-tls\") pod \"maas-api-e2e-ait-5b603d23-7f474cb858-qbpts\" (UID: \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\") " pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:25.850376 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.850262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7614f7af-4710-4724-a15a-9b15ad4ea0d1-openshift-service-ca\") pod \"maas-api-e2e-ait-5b603d23-7f474cb858-qbpts\" (UID: \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\") " pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:25.851017 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.850993    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7614f7af-4710-4724-a15a-9b15ad4ea0d1-openshift-service-ca\") pod \"maas-api-e2e-ait-5b603d23-7f474cb858-qbpts\" (UID: \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\") " pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:25.852880 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.852850    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7614f7af-4710-4724-a15a-9b15ad4ea0d1-maas-api-tls\") pod \"maas-api-e2e-ait-5b603d23-7f474cb858-qbpts\" (UID: \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\") " pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:25.859409 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.859367    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xz8z\" (UniqueName: \"kubernetes.io/projected/7614f7af-4710-4724-a15a-9b15ad4ea0d1-kube-api-access-7xz8z\") pod \"maas-api-e2e-ait-5b603d23-7f474cb858-qbpts\" (UID: \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\") " pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:25.981500 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:25.981465    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:26.136675 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:26.136539    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"]
Jul 06 18:10:26.140663 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:10:26.139602    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7614f7af_4710_4724_a15a_9b15ad4ea0d1.slice/crio-02714505cf7e60c2424a0c373dce4310fc0ca049007984bd0e5e5bf6b3d4c384 WatchSource:0}: Error finding container 02714505cf7e60c2424a0c373dce4310fc0ca049007984bd0e5e5bf6b3d4c384: Status 404 returned error can't find the container with id 02714505cf7e60c2424a0c373dce4310fc0ca049007984bd0e5e5bf6b3d4c384
Jul 06 18:10:26.723530 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:26.723487    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts" event={"ID":"7614f7af-4710-4724-a15a-9b15ad4ea0d1","Type":"ContainerStarted","Data":"dfe98fe2542b9d5029182703a738539b2ffc38a965bfc153a75f4ccd5030b747"}
Jul 06 18:10:26.723958 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:26.723541    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts" event={"ID":"7614f7af-4710-4724-a15a-9b15ad4ea0d1","Type":"ContainerStarted","Data":"02714505cf7e60c2424a0c373dce4310fc0ca049007984bd0e5e5bf6b3d4c384"}
Jul 06 18:10:26.723958 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:26.723571    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:26.740380 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:26.740313    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts" podStartSLOduration=1.235863334 podStartE2EDuration="1.740288772s" totalImagesPullingTime="504.425438ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:10:26.738197778 +0000 UTC m=+1802.331652309" watchObservedRunningTime="2026-07-06 18:10:26.740288772 +0000 UTC m=+1802.333743305"
Jul 06 18:10:32.734087 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:32.734055    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:35.968347 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:35.968309    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"]
Jul 06 18:10:35.968845 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:35.968619    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts" podUID="7614f7af-4710-4724-a15a-9b15ad4ea0d1" containerName="maas-api" containerID="cri-o://dfe98fe2542b9d5029182703a738539b2ffc38a965bfc153a75f4ccd5030b747" gracePeriod=30
Jul 06 18:10:36.233814 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.233736    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:36.247707 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.247669    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7614f7af-4710-4724-a15a-9b15ad4ea0d1-openshift-service-ca\") pod \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\" (UID: \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\") "
Jul 06 18:10:36.247707 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.247709    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xz8z\" (UniqueName: \"kubernetes.io/projected/7614f7af-4710-4724-a15a-9b15ad4ea0d1-kube-api-access-7xz8z\") pod \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\" (UID: \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\") "
Jul 06 18:10:36.248016 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.247777    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7614f7af-4710-4724-a15a-9b15ad4ea0d1-maas-api-tls\") pod \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\" (UID: \"7614f7af-4710-4724-a15a-9b15ad4ea0d1\") "
Jul 06 18:10:36.248124 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.248104    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7614f7af-4710-4724-a15a-9b15ad4ea0d1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7614f7af-4710-4724-a15a-9b15ad4ea0d1" (UID: "7614f7af-4710-4724-a15a-9b15ad4ea0d1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:10:36.250284 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.250235    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7614f7af-4710-4724-a15a-9b15ad4ea0d1-kube-api-access-7xz8z" (OuterVolumeSpecName: "kube-api-access-7xz8z") pod "7614f7af-4710-4724-a15a-9b15ad4ea0d1" (UID: "7614f7af-4710-4724-a15a-9b15ad4ea0d1"). InnerVolumeSpecName "kube-api-access-7xz8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:10:36.250284 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.250241    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7614f7af-4710-4724-a15a-9b15ad4ea0d1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7614f7af-4710-4724-a15a-9b15ad4ea0d1" (UID: "7614f7af-4710-4724-a15a-9b15ad4ea0d1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:10:36.348632 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.348581    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7614f7af-4710-4724-a15a-9b15ad4ea0d1-openshift-service-ca\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:10:36.348632 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.348627    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xz8z\" (UniqueName: \"kubernetes.io/projected/7614f7af-4710-4724-a15a-9b15ad4ea0d1-kube-api-access-7xz8z\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:10:36.348632 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.348638    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7614f7af-4710-4724-a15a-9b15ad4ea0d1-maas-api-tls\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:10:36.780984 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.780941    2581 generic.go:356] "Generic (PLEG): container finished" podID="7614f7af-4710-4724-a15a-9b15ad4ea0d1" containerID="dfe98fe2542b9d5029182703a738539b2ffc38a965bfc153a75f4ccd5030b747" exitCode=0
Jul 06 18:10:36.781167 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.780991    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"
Jul 06 18:10:36.781167 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.781004    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts" event={"ID":"7614f7af-4710-4724-a15a-9b15ad4ea0d1","Type":"ContainerDied","Data":"dfe98fe2542b9d5029182703a738539b2ffc38a965bfc153a75f4ccd5030b747"}
Jul 06 18:10:36.781167 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.781057    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts" event={"ID":"7614f7af-4710-4724-a15a-9b15ad4ea0d1","Type":"ContainerDied","Data":"02714505cf7e60c2424a0c373dce4310fc0ca049007984bd0e5e5bf6b3d4c384"}
Jul 06 18:10:36.781167 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.781088    2581 scope.go:117] "RemoveContainer" containerID="dfe98fe2542b9d5029182703a738539b2ffc38a965bfc153a75f4ccd5030b747"
Jul 06 18:10:36.791084 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.791062    2581 scope.go:117] "RemoveContainer" containerID="dfe98fe2542b9d5029182703a738539b2ffc38a965bfc153a75f4ccd5030b747"
Jul 06 18:10:36.791403 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:10:36.791373    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfe98fe2542b9d5029182703a738539b2ffc38a965bfc153a75f4ccd5030b747\": container with ID starting with dfe98fe2542b9d5029182703a738539b2ffc38a965bfc153a75f4ccd5030b747 not found: ID does not exist" containerID="dfe98fe2542b9d5029182703a738539b2ffc38a965bfc153a75f4ccd5030b747"
Jul 06 18:10:36.791479 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.791408    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfe98fe2542b9d5029182703a738539b2ffc38a965bfc153a75f4ccd5030b747"} err="failed to get container status \"dfe98fe2542b9d5029182703a738539b2ffc38a965bfc153a75f4ccd5030b747\": rpc error: code = NotFound desc = could not find container \"dfe98fe2542b9d5029182703a738539b2ffc38a965bfc153a75f4ccd5030b747\": container with ID starting with dfe98fe2542b9d5029182703a738539b2ffc38a965bfc153a75f4ccd5030b747 not found: ID does not exist"
Jul 06 18:10:36.802109 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.802075    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"]
Jul 06 18:10:36.805751 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.805721    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-5b603d23-7f474cb858-qbpts"]
Jul 06 18:10:36.913362 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:36.913328    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7614f7af-4710-4724-a15a-9b15ad4ea0d1" path="/var/lib/kubelet/pods/7614f7af-4710-4724-a15a-9b15ad4ea0d1/volumes"
Jul 06 18:10:59.201441 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.201398    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"]
Jul 06 18:10:59.202079 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.201819    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7614f7af-4710-4724-a15a-9b15ad4ea0d1" containerName="maas-api"
Jul 06 18:10:59.202079 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.201838    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7614f7af-4710-4724-a15a-9b15ad4ea0d1" containerName="maas-api"
Jul 06 18:10:59.202079 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.201996    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7614f7af-4710-4724-a15a-9b15ad4ea0d1" containerName="maas-api"
Jul 06 18:10:59.211199 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.211165    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:10:59.214256 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.213968    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"]
Jul 06 18:10:59.214256 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.213969    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-34249437-serving-cert\""
Jul 06 18:10:59.243008 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.242965    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2037cc7e-4d84-4354-972a-2e864ca8e236-openshift-service-ca\") pod \"maas-api-e2e-derive-34249437-bdfd76c58-vm2xf\" (UID: \"2037cc7e-4d84-4354-972a-2e864ca8e236\") " pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:10:59.243200 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.243025    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8rbl\" (UniqueName: \"kubernetes.io/projected/2037cc7e-4d84-4354-972a-2e864ca8e236-kube-api-access-q8rbl\") pod \"maas-api-e2e-derive-34249437-bdfd76c58-vm2xf\" (UID: \"2037cc7e-4d84-4354-972a-2e864ca8e236\") " pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:10:59.243200 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.243097    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2037cc7e-4d84-4354-972a-2e864ca8e236-maas-api-tls\") pod \"maas-api-e2e-derive-34249437-bdfd76c58-vm2xf\" (UID: \"2037cc7e-4d84-4354-972a-2e864ca8e236\") " pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:10:59.343943 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.343876    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2037cc7e-4d84-4354-972a-2e864ca8e236-openshift-service-ca\") pod \"maas-api-e2e-derive-34249437-bdfd76c58-vm2xf\" (UID: \"2037cc7e-4d84-4354-972a-2e864ca8e236\") " pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:10:59.343943 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.343951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8rbl\" (UniqueName: \"kubernetes.io/projected/2037cc7e-4d84-4354-972a-2e864ca8e236-kube-api-access-q8rbl\") pod \"maas-api-e2e-derive-34249437-bdfd76c58-vm2xf\" (UID: \"2037cc7e-4d84-4354-972a-2e864ca8e236\") " pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:10:59.344230 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.343971    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2037cc7e-4d84-4354-972a-2e864ca8e236-maas-api-tls\") pod \"maas-api-e2e-derive-34249437-bdfd76c58-vm2xf\" (UID: \"2037cc7e-4d84-4354-972a-2e864ca8e236\") " pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:10:59.344230 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:10:59.344101    2581 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-34249437-serving-cert: secret "maas-api-e2e-derive-34249437-serving-cert" not found
Jul 06 18:10:59.344230 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:10:59.344155    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2037cc7e-4d84-4354-972a-2e864ca8e236-maas-api-tls podName:2037cc7e-4d84-4354-972a-2e864ca8e236 nodeName:}" failed. No retries permitted until 2026-07-06 18:10:59.84413913 +0000 UTC m=+1835.437593637 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2037cc7e-4d84-4354-972a-2e864ca8e236-maas-api-tls") pod "maas-api-e2e-derive-34249437-bdfd76c58-vm2xf" (UID: "2037cc7e-4d84-4354-972a-2e864ca8e236") : secret "maas-api-e2e-derive-34249437-serving-cert" not found
Jul 06 18:10:59.344682 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.344654    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2037cc7e-4d84-4354-972a-2e864ca8e236-openshift-service-ca\") pod \"maas-api-e2e-derive-34249437-bdfd76c58-vm2xf\" (UID: \"2037cc7e-4d84-4354-972a-2e864ca8e236\") " pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:10:59.352320 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.352290    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8rbl\" (UniqueName: \"kubernetes.io/projected/2037cc7e-4d84-4354-972a-2e864ca8e236-kube-api-access-q8rbl\") pod \"maas-api-e2e-derive-34249437-bdfd76c58-vm2xf\" (UID: \"2037cc7e-4d84-4354-972a-2e864ca8e236\") " pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:10:59.849690 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.849650    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2037cc7e-4d84-4354-972a-2e864ca8e236-maas-api-tls\") pod \"maas-api-e2e-derive-34249437-bdfd76c58-vm2xf\" (UID: \"2037cc7e-4d84-4354-972a-2e864ca8e236\") " pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:10:59.852387 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:10:59.852352    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2037cc7e-4d84-4354-972a-2e864ca8e236-maas-api-tls\") pod \"maas-api-e2e-derive-34249437-bdfd76c58-vm2xf\" (UID: \"2037cc7e-4d84-4354-972a-2e864ca8e236\") " pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:11:00.126214 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:00.126110    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:11:00.300335 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:00.300295    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"]
Jul 06 18:11:00.302948 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:11:00.302921    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2037cc7e_4d84_4354_972a_2e864ca8e236.slice/crio-b34d86670b95b40d55f4d7a0a6d4feb0fcb154e3f09801cdaa29b8976205aa41 WatchSource:0}: Error finding container b34d86670b95b40d55f4d7a0a6d4feb0fcb154e3f09801cdaa29b8976205aa41: Status 404 returned error can't find the container with id b34d86670b95b40d55f4d7a0a6d4feb0fcb154e3f09801cdaa29b8976205aa41
Jul 06 18:11:00.897830 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:00.897724    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf" event={"ID":"2037cc7e-4d84-4354-972a-2e864ca8e236","Type":"ContainerStarted","Data":"13100fc16178456a7c7597ee9a47002e6f4af4d353151d638d2eaf24f1d767d1"}
Jul 06 18:11:00.897830 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:00.897785    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf" event={"ID":"2037cc7e-4d84-4354-972a-2e864ca8e236","Type":"ContainerStarted","Data":"b34d86670b95b40d55f4d7a0a6d4feb0fcb154e3f09801cdaa29b8976205aa41"}
Jul 06 18:11:00.898095 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:00.897830    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:11:00.917259 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:00.917191    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf" podStartSLOduration=1.619266023 podStartE2EDuration="1.91716747s" totalImagesPullingTime="297.901447ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:11:00.91409542 +0000 UTC m=+1836.507549952" watchObservedRunningTime="2026-07-06 18:11:00.91716747 +0000 UTC m=+1836.510622005"
Jul 06 18:11:03.746458 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:03.746410    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"]
Jul 06 18:11:03.746994 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:03.746668    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf" podUID="2037cc7e-4d84-4354-972a-2e864ca8e236" containerName="maas-api" containerID="cri-o://13100fc16178456a7c7597ee9a47002e6f4af4d353151d638d2eaf24f1d767d1" gracePeriod=30
Jul 06 18:11:03.914126 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:03.914075    2581 generic.go:356] "Generic (PLEG): container finished" podID="2037cc7e-4d84-4354-972a-2e864ca8e236" containerID="13100fc16178456a7c7597ee9a47002e6f4af4d353151d638d2eaf24f1d767d1" exitCode=0
Jul 06 18:11:03.914311 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:03.914144    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf" event={"ID":"2037cc7e-4d84-4354-972a-2e864ca8e236","Type":"ContainerDied","Data":"13100fc16178456a7c7597ee9a47002e6f4af4d353151d638d2eaf24f1d767d1"}
Jul 06 18:11:03.998698 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:03.998624    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:11:04.089056 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:04.089017    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2037cc7e-4d84-4354-972a-2e864ca8e236-openshift-service-ca\") pod \"2037cc7e-4d84-4354-972a-2e864ca8e236\" (UID: \"2037cc7e-4d84-4354-972a-2e864ca8e236\") "
Jul 06 18:11:04.089239 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:04.089104    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8rbl\" (UniqueName: \"kubernetes.io/projected/2037cc7e-4d84-4354-972a-2e864ca8e236-kube-api-access-q8rbl\") pod \"2037cc7e-4d84-4354-972a-2e864ca8e236\" (UID: \"2037cc7e-4d84-4354-972a-2e864ca8e236\") "
Jul 06 18:11:04.089239 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:04.089146    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2037cc7e-4d84-4354-972a-2e864ca8e236-maas-api-tls\") pod \"2037cc7e-4d84-4354-972a-2e864ca8e236\" (UID: \"2037cc7e-4d84-4354-972a-2e864ca8e236\") "
Jul 06 18:11:04.089459 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:04.089436    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2037cc7e-4d84-4354-972a-2e864ca8e236-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2037cc7e-4d84-4354-972a-2e864ca8e236" (UID: "2037cc7e-4d84-4354-972a-2e864ca8e236"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:11:04.091255 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:04.091223    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2037cc7e-4d84-4354-972a-2e864ca8e236-kube-api-access-q8rbl" (OuterVolumeSpecName: "kube-api-access-q8rbl") pod "2037cc7e-4d84-4354-972a-2e864ca8e236" (UID: "2037cc7e-4d84-4354-972a-2e864ca8e236"). InnerVolumeSpecName "kube-api-access-q8rbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:11:04.091381 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:04.091304    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2037cc7e-4d84-4354-972a-2e864ca8e236-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2037cc7e-4d84-4354-972a-2e864ca8e236" (UID: "2037cc7e-4d84-4354-972a-2e864ca8e236"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:11:04.190233 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:04.190188    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2037cc7e-4d84-4354-972a-2e864ca8e236-openshift-service-ca\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:11:04.190233 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:04.190222    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8rbl\" (UniqueName: \"kubernetes.io/projected/2037cc7e-4d84-4354-972a-2e864ca8e236-kube-api-access-q8rbl\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:11:04.190233 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:04.190234    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2037cc7e-4d84-4354-972a-2e864ca8e236-maas-api-tls\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:11:04.919928 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:04.919872    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf" event={"ID":"2037cc7e-4d84-4354-972a-2e864ca8e236","Type":"ContainerDied","Data":"b34d86670b95b40d55f4d7a0a6d4feb0fcb154e3f09801cdaa29b8976205aa41"}
Jul 06 18:11:04.920390 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:04.919912    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"
Jul 06 18:11:04.920390 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:04.919958    2581 scope.go:117] "RemoveContainer" containerID="13100fc16178456a7c7597ee9a47002e6f4af4d353151d638d2eaf24f1d767d1"
Jul 06 18:11:04.937711 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:04.937674    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"]
Jul 06 18:11:04.941169 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:04.941139    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-34249437-bdfd76c58-vm2xf"]
Jul 06 18:11:06.913488 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:11:06.913452    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2037cc7e-4d84-4354-972a-2e864ca8e236" path="/var/lib/kubelet/pods/2037cc7e-4d84-4354-972a-2e864ca8e236/volumes"
Jul 06 18:12:47.438761 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.438720    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"]
Jul 06 18:12:47.441176 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.439162    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2037cc7e-4d84-4354-972a-2e864ca8e236" containerName="maas-api"
Jul 06 18:12:47.441176 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.439176    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2037cc7e-4d84-4354-972a-2e864ca8e236" containerName="maas-api"
Jul 06 18:12:47.441176 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.439243    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2037cc7e-4d84-4354-972a-2e864ca8e236" containerName="maas-api"
Jul 06 18:12:47.442053 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.442031    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:47.445363 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.445322    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-86957ffb-serving-cert\""
Jul 06 18:12:47.455562 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.455531    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"]
Jul 06 18:12:47.524109 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.524066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c7257da-1406-44f6-bf6c-21f698813f87-openshift-service-ca\") pod \"maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp\" (UID: \"4c7257da-1406-44f6-bf6c-21f698813f87\") " pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:47.524298 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.524139    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zg25\" (UniqueName: \"kubernetes.io/projected/4c7257da-1406-44f6-bf6c-21f698813f87-kube-api-access-8zg25\") pod \"maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp\" (UID: \"4c7257da-1406-44f6-bf6c-21f698813f87\") " pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:47.524298 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.524170    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c7257da-1406-44f6-bf6c-21f698813f87-maas-api-tls\") pod \"maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp\" (UID: \"4c7257da-1406-44f6-bf6c-21f698813f87\") " pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:47.625146 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.625098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c7257da-1406-44f6-bf6c-21f698813f87-maas-api-tls\") pod \"maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp\" (UID: \"4c7257da-1406-44f6-bf6c-21f698813f87\") " pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:47.625370 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.625189    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c7257da-1406-44f6-bf6c-21f698813f87-openshift-service-ca\") pod \"maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp\" (UID: \"4c7257da-1406-44f6-bf6c-21f698813f87\") " pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:47.625370 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.625249    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zg25\" (UniqueName: \"kubernetes.io/projected/4c7257da-1406-44f6-bf6c-21f698813f87-kube-api-access-8zg25\") pod \"maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp\" (UID: \"4c7257da-1406-44f6-bf6c-21f698813f87\") " pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:47.625370 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:12:47.625255    2581 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-86957ffb-serving-cert: secret "maas-api-e2e-mt-86957ffb-serving-cert" not found
Jul 06 18:12:47.625370 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:12:47.625324    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c7257da-1406-44f6-bf6c-21f698813f87-maas-api-tls podName:4c7257da-1406-44f6-bf6c-21f698813f87 nodeName:}" failed. No retries permitted until 2026-07-06 18:12:48.125306329 +0000 UTC m=+1943.718760837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c7257da-1406-44f6-bf6c-21f698813f87-maas-api-tls") pod "maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp" (UID: "4c7257da-1406-44f6-bf6c-21f698813f87") : secret "maas-api-e2e-mt-86957ffb-serving-cert" not found
Jul 06 18:12:47.625817 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.625795    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c7257da-1406-44f6-bf6c-21f698813f87-openshift-service-ca\") pod \"maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp\" (UID: \"4c7257da-1406-44f6-bf6c-21f698813f87\") " pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:47.646130 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:47.646092    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zg25\" (UniqueName: \"kubernetes.io/projected/4c7257da-1406-44f6-bf6c-21f698813f87-kube-api-access-8zg25\") pod \"maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp\" (UID: \"4c7257da-1406-44f6-bf6c-21f698813f87\") " pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:48.130748 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:48.130703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c7257da-1406-44f6-bf6c-21f698813f87-maas-api-tls\") pod \"maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp\" (UID: \"4c7257da-1406-44f6-bf6c-21f698813f87\") " pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:48.133424 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:48.133391    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c7257da-1406-44f6-bf6c-21f698813f87-maas-api-tls\") pod \"maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp\" (UID: \"4c7257da-1406-44f6-bf6c-21f698813f87\") " pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:48.355148 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:48.355096    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:48.510941 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:48.510910    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"]
Jul 06 18:12:48.513434 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:12:48.513403    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c7257da_1406_44f6_bf6c_21f698813f87.slice/crio-1cfaa9ea36aa6c8a030083292fc5c1d51de9545c27c161dfb882955e081e870a WatchSource:0}: Error finding container 1cfaa9ea36aa6c8a030083292fc5c1d51de9545c27c161dfb882955e081e870a: Status 404 returned error can't find the container with id 1cfaa9ea36aa6c8a030083292fc5c1d51de9545c27c161dfb882955e081e870a
Jul 06 18:12:49.435060 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:49.435014    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp" event={"ID":"4c7257da-1406-44f6-bf6c-21f698813f87","Type":"ContainerStarted","Data":"48c443b5f39f9872d79f13c1a5fbe625a2c5b442be6de64f020a12a101f9b703"}
Jul 06 18:12:49.435272 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:49.435075    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp" event={"ID":"4c7257da-1406-44f6-bf6c-21f698813f87","Type":"ContainerStarted","Data":"1cfaa9ea36aa6c8a030083292fc5c1d51de9545c27c161dfb882955e081e870a"}
Jul 06 18:12:49.435272 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:49.435193    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:49.453123 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:49.453064    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp" podStartSLOduration=2.048357148 podStartE2EDuration="2.453045302s" totalImagesPullingTime="404.688154ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:12:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:12:49.451193808 +0000 UTC m=+1945.044648339" watchObservedRunningTime="2026-07-06 18:12:49.453045302 +0000 UTC m=+1945.046499831"
Jul 06 18:12:55.445959 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:55.445928    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:59.460122 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:59.460046    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"]
Jul 06 18:12:59.460735 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:59.460306    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp" podUID="4c7257da-1406-44f6-bf6c-21f698813f87" containerName="maas-api" containerID="cri-o://48c443b5f39f9872d79f13c1a5fbe625a2c5b442be6de64f020a12a101f9b703" gracePeriod=30
Jul 06 18:12:59.709321 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:59.709282    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:12:59.833948 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:59.833839    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zg25\" (UniqueName: \"kubernetes.io/projected/4c7257da-1406-44f6-bf6c-21f698813f87-kube-api-access-8zg25\") pod \"4c7257da-1406-44f6-bf6c-21f698813f87\" (UID: \"4c7257da-1406-44f6-bf6c-21f698813f87\") "
Jul 06 18:12:59.833948 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:59.833948    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c7257da-1406-44f6-bf6c-21f698813f87-openshift-service-ca\") pod \"4c7257da-1406-44f6-bf6c-21f698813f87\" (UID: \"4c7257da-1406-44f6-bf6c-21f698813f87\") "
Jul 06 18:12:59.834200 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:59.834023    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c7257da-1406-44f6-bf6c-21f698813f87-maas-api-tls\") pod \"4c7257da-1406-44f6-bf6c-21f698813f87\" (UID: \"4c7257da-1406-44f6-bf6c-21f698813f87\") "
Jul 06 18:12:59.834334 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:59.834301    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c7257da-1406-44f6-bf6c-21f698813f87-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c7257da-1406-44f6-bf6c-21f698813f87" (UID: "4c7257da-1406-44f6-bf6c-21f698813f87"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:12:59.836204 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:59.836171    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c7257da-1406-44f6-bf6c-21f698813f87-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c7257da-1406-44f6-bf6c-21f698813f87" (UID: "4c7257da-1406-44f6-bf6c-21f698813f87"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:12:59.836312 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:59.836171    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c7257da-1406-44f6-bf6c-21f698813f87-kube-api-access-8zg25" (OuterVolumeSpecName: "kube-api-access-8zg25") pod "4c7257da-1406-44f6-bf6c-21f698813f87" (UID: "4c7257da-1406-44f6-bf6c-21f698813f87"). InnerVolumeSpecName "kube-api-access-8zg25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:12:59.935033 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:59.934996    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c7257da-1406-44f6-bf6c-21f698813f87-maas-api-tls\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:12:59.935033 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:59.935032    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zg25\" (UniqueName: \"kubernetes.io/projected/4c7257da-1406-44f6-bf6c-21f698813f87-kube-api-access-8zg25\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:12:59.935247 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:12:59.935048    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c7257da-1406-44f6-bf6c-21f698813f87-openshift-service-ca\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:13:00.495740 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:00.495702    2581 generic.go:356] "Generic (PLEG): container finished" podID="4c7257da-1406-44f6-bf6c-21f698813f87" containerID="48c443b5f39f9872d79f13c1a5fbe625a2c5b442be6de64f020a12a101f9b703" exitCode=0
Jul 06 18:13:00.496169 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:00.495752    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"
Jul 06 18:13:00.496169 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:00.495753    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp" event={"ID":"4c7257da-1406-44f6-bf6c-21f698813f87","Type":"ContainerDied","Data":"48c443b5f39f9872d79f13c1a5fbe625a2c5b442be6de64f020a12a101f9b703"}
Jul 06 18:13:00.496169 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:00.495885    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp" event={"ID":"4c7257da-1406-44f6-bf6c-21f698813f87","Type":"ContainerDied","Data":"1cfaa9ea36aa6c8a030083292fc5c1d51de9545c27c161dfb882955e081e870a"}
Jul 06 18:13:00.496169 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:00.495939    2581 scope.go:117] "RemoveContainer" containerID="48c443b5f39f9872d79f13c1a5fbe625a2c5b442be6de64f020a12a101f9b703"
Jul 06 18:13:00.506270 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:00.506248    2581 scope.go:117] "RemoveContainer" containerID="48c443b5f39f9872d79f13c1a5fbe625a2c5b442be6de64f020a12a101f9b703"
Jul 06 18:13:00.506639 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:13:00.506617    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48c443b5f39f9872d79f13c1a5fbe625a2c5b442be6de64f020a12a101f9b703\": container with ID starting with 48c443b5f39f9872d79f13c1a5fbe625a2c5b442be6de64f020a12a101f9b703 not found: ID does not exist" containerID="48c443b5f39f9872d79f13c1a5fbe625a2c5b442be6de64f020a12a101f9b703"
Jul 06 18:13:00.506695 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:00.506650    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48c443b5f39f9872d79f13c1a5fbe625a2c5b442be6de64f020a12a101f9b703"} err="failed to get container status \"48c443b5f39f9872d79f13c1a5fbe625a2c5b442be6de64f020a12a101f9b703\": rpc error: code = NotFound desc = could not find container \"48c443b5f39f9872d79f13c1a5fbe625a2c5b442be6de64f020a12a101f9b703\": container with ID starting with 48c443b5f39f9872d79f13c1a5fbe625a2c5b442be6de64f020a12a101f9b703 not found: ID does not exist"
Jul 06 18:13:00.517928 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:00.517873    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"]
Jul 06 18:13:00.523320 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:00.523284    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-86957ffb-bf87695c7-pqtnp"]
Jul 06 18:13:00.912159 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:00.912071    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c7257da-1406-44f6-bf6c-21f698813f87" path="/var/lib/kubelet/pods/4c7257da-1406-44f6-bf6c-21f698813f87/volumes"
Jul 06 18:13:28.626966 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.626922    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"]
Jul 06 18:13:28.627430 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.627305    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c7257da-1406-44f6-bf6c-21f698813f87" containerName="maas-api"
Jul 06 18:13:28.627430 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.627316    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c7257da-1406-44f6-bf6c-21f698813f87" containerName="maas-api"
Jul 06 18:13:28.627430 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.627384    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c7257da-1406-44f6-bf6c-21f698813f87" containerName="maas-api"
Jul 06 18:13:28.630366 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.630343    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:13:28.634833 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.634795    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-14d0a8da-serving-cert\""
Jul 06 18:13:28.643145 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.642823    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"]
Jul 06 18:13:28.692830 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.692790    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/384971a1-d1d0-4b28-8ab3-4ee328fddcea-openshift-service-ca\") pod \"maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8\" (UID: \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\") " pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:13:28.692830 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.692836    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq2gw\" (UniqueName: \"kubernetes.io/projected/384971a1-d1d0-4b28-8ab3-4ee328fddcea-kube-api-access-jq2gw\") pod \"maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8\" (UID: \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\") " pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:13:28.693091 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.692860    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/384971a1-d1d0-4b28-8ab3-4ee328fddcea-maas-api-tls\") pod \"maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8\" (UID: \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\") " pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:13:28.793626 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.793586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/384971a1-d1d0-4b28-8ab3-4ee328fddcea-openshift-service-ca\") pod \"maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8\" (UID: \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\") " pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:13:28.793814 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.793638    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq2gw\" (UniqueName: \"kubernetes.io/projected/384971a1-d1d0-4b28-8ab3-4ee328fddcea-kube-api-access-jq2gw\") pod \"maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8\" (UID: \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\") " pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:13:28.793814 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.793675    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/384971a1-d1d0-4b28-8ab3-4ee328fddcea-maas-api-tls\") pod \"maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8\" (UID: \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\") " pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:13:28.793972 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:13:28.793839    2581 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-14d0a8da-serving-cert: secret "maas-api-e2e-mt-14d0a8da-serving-cert" not found
Jul 06 18:13:28.793972 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:13:28.793960    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/384971a1-d1d0-4b28-8ab3-4ee328fddcea-maas-api-tls podName:384971a1-d1d0-4b28-8ab3-4ee328fddcea nodeName:}" failed. No retries permitted until 2026-07-06 18:13:29.293933787 +0000 UTC m=+1984.887388295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/384971a1-d1d0-4b28-8ab3-4ee328fddcea-maas-api-tls") pod "maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8" (UID: "384971a1-d1d0-4b28-8ab3-4ee328fddcea") : secret "maas-api-e2e-mt-14d0a8da-serving-cert" not found
Jul 06 18:13:28.794456 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.794434    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/384971a1-d1d0-4b28-8ab3-4ee328fddcea-openshift-service-ca\") pod \"maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8\" (UID: \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\") " pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:13:28.802790 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:28.802752    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq2gw\" (UniqueName: \"kubernetes.io/projected/384971a1-d1d0-4b28-8ab3-4ee328fddcea-kube-api-access-jq2gw\") pod \"maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8\" (UID: \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\") " pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:13:29.298405 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:29.298366    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/384971a1-d1d0-4b28-8ab3-4ee328fddcea-maas-api-tls\") pod \"maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8\" (UID: \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\") " pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:13:29.301061 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:29.301037    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/384971a1-d1d0-4b28-8ab3-4ee328fddcea-maas-api-tls\") pod \"maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8\" (UID: \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\") " pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:13:29.548324 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:29.548286    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:13:29.699726 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:29.699691    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"]
Jul 06 18:13:29.702095 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:13:29.702068    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod384971a1_d1d0_4b28_8ab3_4ee328fddcea.slice/crio-bdbb5c7474d96fbb948111cb0fa1b46233ac313c5f79190553e36a16af547ccb WatchSource:0}: Error finding container bdbb5c7474d96fbb948111cb0fa1b46233ac313c5f79190553e36a16af547ccb: Status 404 returned error can't find the container with id bdbb5c7474d96fbb948111cb0fa1b46233ac313c5f79190553e36a16af547ccb
Jul 06 18:13:30.662665 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:30.662623    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8" event={"ID":"384971a1-d1d0-4b28-8ab3-4ee328fddcea","Type":"ContainerStarted","Data":"eff5906515678bb24e8ff3100d5a7ae619a2514c9b1e111977e64edf34630c97"}
Jul 06 18:13:30.662916 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:30.662713    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8" event={"ID":"384971a1-d1d0-4b28-8ab3-4ee328fddcea","Type":"ContainerStarted","Data":"bdbb5c7474d96fbb948111cb0fa1b46233ac313c5f79190553e36a16af547ccb"}
Jul 06 18:13:30.662916 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:30.662855    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:13:30.681409 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:30.681341    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8" podStartSLOduration=2.248111154 podStartE2EDuration="2.681322659s" totalImagesPullingTime="433.211505ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:13:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:13:30.67926882 +0000 UTC m=+1986.272723351" watchObservedRunningTime="2026-07-06 18:13:30.681322659 +0000 UTC m=+1986.274777189"
Jul 06 18:13:36.674809 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:13:36.674780    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:14:03.198589 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.198551    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"]
Jul 06 18:14:03.202370 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.202343    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.204875 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.204851    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-14d0a8da\"/\"kube-root-ca.crt\""
Jul 06 18:14:03.206238 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.206217    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-14d0a8da\"/\"openshift-service-ca.crt\""
Jul 06 18:14:03.206394 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.206290    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-14d0a8da\"/\"default-dockercfg-dm6mk\""
Jul 06 18:14:03.206394 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.206344    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-14d0a8da\"/\"e2e-lifecycle-model-14d0a8da-kserve-self-signed-certs\""
Jul 06 18:14:03.212932 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.212886    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"]
Jul 06 18:14:03.296655 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.296609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-tmp-dir\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.296840 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.296716    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-home\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.296840 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.296761    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6z8g\" (UniqueName: \"kubernetes.io/projected/b13f5d06-aebb-4761-b06c-95ad8d82d77c-kube-api-access-n6z8g\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.296840 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.296797    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-model-cache\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.296840 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.296840    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-dshm\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.297021 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.296858    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b13f5d06-aebb-4761-b06c-95ad8d82d77c-tls-certs\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.398056 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.398011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-home\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.398056 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.398057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6z8g\" (UniqueName: \"kubernetes.io/projected/b13f5d06-aebb-4761-b06c-95ad8d82d77c-kube-api-access-n6z8g\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.398293 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.398098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-model-cache\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.398293 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.398142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-dshm\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.398371 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.398345    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b13f5d06-aebb-4761-b06c-95ad8d82d77c-tls-certs\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.398419 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.398409    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-tmp-dir\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.398508 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.398487    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-home\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.398578 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.398537    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-model-cache\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.398758 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.398734    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-tmp-dir\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.400429 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.400406    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-dshm\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.400807 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.400785    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b13f5d06-aebb-4761-b06c-95ad8d82d77c-tls-certs\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.405980 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.405954    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6z8g\" (UniqueName: \"kubernetes.io/projected/b13f5d06-aebb-4761-b06c-95ad8d82d77c-kube-api-access-n6z8g\") pod \"e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") " pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.515734 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.515618    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:03.656656 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.656628    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"]
Jul 06 18:14:03.659442 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:14:03.659412    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb13f5d06_aebb_4761_b06c_95ad8d82d77c.slice/crio-dfa89a31c199244d6b1b1c67f39fa11af6a7766f7362c358027f227fbbc1dcfa WatchSource:0}: Error finding container dfa89a31c199244d6b1b1c67f39fa11af6a7766f7362c358027f227fbbc1dcfa: Status 404 returned error can't find the container with id dfa89a31c199244d6b1b1c67f39fa11af6a7766f7362c358027f227fbbc1dcfa
Jul 06 18:14:03.830269 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.830166    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99" event={"ID":"b13f5d06-aebb-4761-b06c-95ad8d82d77c","Type":"ContainerStarted","Data":"36d7c18f0e0d97b38793b181ae73204b68ec1946fedc6d08ae49cb45544a0971"}
Jul 06 18:14:03.830269 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.830215    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99" event={"ID":"b13f5d06-aebb-4761-b06c-95ad8d82d77c","Type":"ContainerStarted","Data":"dfa89a31c199244d6b1b1c67f39fa11af6a7766f7362c358027f227fbbc1dcfa"}
Jul 06 18:14:03.846568 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:03.846497    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99" podStartSLOduration=0.846475133 podStartE2EDuration="846.475133ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:14:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:14:03.845039377 +0000 UTC m=+2019.438493907" watchObservedRunningTime="2026-07-06 18:14:03.846475133 +0000 UTC m=+2019.439929664"
Jul 06 18:14:13.516042 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:13.515978    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:13.516042 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:13.516050    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:13.528729 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:13.528697    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:13.892215 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:13.892119    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:21.097273 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.097234    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"]
Jul 06 18:14:21.097857 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.097462    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8" podUID="384971a1-d1d0-4b28-8ab3-4ee328fddcea" containerName="maas-api" containerID="cri-o://eff5906515678bb24e8ff3100d5a7ae619a2514c9b1e111977e64edf34630c97" gracePeriod=30
Jul 06 18:14:21.354362 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.354270    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:14:21.466658 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.466618    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/384971a1-d1d0-4b28-8ab3-4ee328fddcea-maas-api-tls\") pod \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\" (UID: \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\") "
Jul 06 18:14:21.466816 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.466722    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jq2gw\" (UniqueName: \"kubernetes.io/projected/384971a1-d1d0-4b28-8ab3-4ee328fddcea-kube-api-access-jq2gw\") pod \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\" (UID: \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\") "
Jul 06 18:14:21.466964 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.466889    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/384971a1-d1d0-4b28-8ab3-4ee328fddcea-openshift-service-ca\") pod \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\" (UID: \"384971a1-d1d0-4b28-8ab3-4ee328fddcea\") "
Jul 06 18:14:21.467199 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.467172    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/384971a1-d1d0-4b28-8ab3-4ee328fddcea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "384971a1-d1d0-4b28-8ab3-4ee328fddcea" (UID: "384971a1-d1d0-4b28-8ab3-4ee328fddcea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:14:21.468910 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.468872    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/384971a1-d1d0-4b28-8ab3-4ee328fddcea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "384971a1-d1d0-4b28-8ab3-4ee328fddcea" (UID: "384971a1-d1d0-4b28-8ab3-4ee328fddcea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:14:21.469115 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.469090    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/384971a1-d1d0-4b28-8ab3-4ee328fddcea-kube-api-access-jq2gw" (OuterVolumeSpecName: "kube-api-access-jq2gw") pod "384971a1-d1d0-4b28-8ab3-4ee328fddcea" (UID: "384971a1-d1d0-4b28-8ab3-4ee328fddcea"). InnerVolumeSpecName "kube-api-access-jq2gw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:14:21.567857 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.567820    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/384971a1-d1d0-4b28-8ab3-4ee328fddcea-maas-api-tls\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:14:21.567857 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.567851    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq2gw\" (UniqueName: \"kubernetes.io/projected/384971a1-d1d0-4b28-8ab3-4ee328fddcea-kube-api-access-jq2gw\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:14:21.567857 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.567862    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/384971a1-d1d0-4b28-8ab3-4ee328fddcea-openshift-service-ca\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:14:21.918401 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.918365    2581 generic.go:356] "Generic (PLEG): container finished" podID="384971a1-d1d0-4b28-8ab3-4ee328fddcea" containerID="eff5906515678bb24e8ff3100d5a7ae619a2514c9b1e111977e64edf34630c97" exitCode=0
Jul 06 18:14:21.918597 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.918412    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"
Jul 06 18:14:21.918597 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.918452    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8" event={"ID":"384971a1-d1d0-4b28-8ab3-4ee328fddcea","Type":"ContainerDied","Data":"eff5906515678bb24e8ff3100d5a7ae619a2514c9b1e111977e64edf34630c97"}
Jul 06 18:14:21.918597 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.918513    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8" event={"ID":"384971a1-d1d0-4b28-8ab3-4ee328fddcea","Type":"ContainerDied","Data":"bdbb5c7474d96fbb948111cb0fa1b46233ac313c5f79190553e36a16af547ccb"}
Jul 06 18:14:21.918597 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.918545    2581 scope.go:117] "RemoveContainer" containerID="eff5906515678bb24e8ff3100d5a7ae619a2514c9b1e111977e64edf34630c97"
Jul 06 18:14:21.928250 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.928231    2581 scope.go:117] "RemoveContainer" containerID="eff5906515678bb24e8ff3100d5a7ae619a2514c9b1e111977e64edf34630c97"
Jul 06 18:14:21.928611 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:14:21.928564    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eff5906515678bb24e8ff3100d5a7ae619a2514c9b1e111977e64edf34630c97\": container with ID starting with eff5906515678bb24e8ff3100d5a7ae619a2514c9b1e111977e64edf34630c97 not found: ID does not exist" containerID="eff5906515678bb24e8ff3100d5a7ae619a2514c9b1e111977e64edf34630c97"
Jul 06 18:14:21.928611 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.928596    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eff5906515678bb24e8ff3100d5a7ae619a2514c9b1e111977e64edf34630c97"} err="failed to get container status \"eff5906515678bb24e8ff3100d5a7ae619a2514c9b1e111977e64edf34630c97\": rpc error: code = NotFound desc = could not find container \"eff5906515678bb24e8ff3100d5a7ae619a2514c9b1e111977e64edf34630c97\": container with ID starting with eff5906515678bb24e8ff3100d5a7ae619a2514c9b1e111977e64edf34630c97 not found: ID does not exist"
Jul 06 18:14:21.940435 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.940396    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"]
Jul 06 18:14:21.943468 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:21.943440    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-14d0a8da-fccff4f7f-frhc8"]
Jul 06 18:14:22.912201 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:22.912169    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="384971a1-d1d0-4b28-8ab3-4ee328fddcea" path="/var/lib/kubelet/pods/384971a1-d1d0-4b28-8ab3-4ee328fddcea/volumes"
Jul 06 18:14:31.848099 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:31.848055    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"]
Jul 06 18:14:46.869246 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:46.869130    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99" podUID="b13f5d06-aebb-4761-b06c-95ad8d82d77c" containerName="main" containerID="cri-o://36d7c18f0e0d97b38793b181ae73204b68ec1946fedc6d08ae49cb45544a0971" gracePeriod=45
Jul 06 18:14:47.055618 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.055576    2581 generic.go:356] "Generic (PLEG): container finished" podID="b13f5d06-aebb-4761-b06c-95ad8d82d77c" containerID="36d7c18f0e0d97b38793b181ae73204b68ec1946fedc6d08ae49cb45544a0971" exitCode=0
Jul 06 18:14:47.055811 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.055692    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99" event={"ID":"b13f5d06-aebb-4761-b06c-95ad8d82d77c","Type":"ContainerDied","Data":"36d7c18f0e0d97b38793b181ae73204b68ec1946fedc6d08ae49cb45544a0971"}
Jul 06 18:14:47.133784 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.133711    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:47.194072 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.194013    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-dshm\") pod \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") "
Jul 06 18:14:47.194257 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.194098    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-tmp-dir\") pod \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") "
Jul 06 18:14:47.194257 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.194124    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-home\") pod \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") "
Jul 06 18:14:47.194257 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.194202    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-model-cache\") pod \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") "
Jul 06 18:14:47.194257 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.194245    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b13f5d06-aebb-4761-b06c-95ad8d82d77c-tls-certs\") pod \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") "
Jul 06 18:14:47.194555 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.194286    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6z8g\" (UniqueName: \"kubernetes.io/projected/b13f5d06-aebb-4761-b06c-95ad8d82d77c-kube-api-access-n6z8g\") pod \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\" (UID: \"b13f5d06-aebb-4761-b06c-95ad8d82d77c\") "
Jul 06 18:14:47.194555 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.194396    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b13f5d06-aebb-4761-b06c-95ad8d82d77c" (UID: "b13f5d06-aebb-4761-b06c-95ad8d82d77c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:14:47.194555 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.194407    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-home" (OuterVolumeSpecName: "home") pod "b13f5d06-aebb-4761-b06c-95ad8d82d77c" (UID: "b13f5d06-aebb-4761-b06c-95ad8d82d77c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:14:47.194555 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.194485    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-model-cache" (OuterVolumeSpecName: "model-cache") pod "b13f5d06-aebb-4761-b06c-95ad8d82d77c" (UID: "b13f5d06-aebb-4761-b06c-95ad8d82d77c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:14:47.194555 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.194527    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-tmp-dir\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:14:47.194555 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.194540    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-home\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:14:47.196550 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.196520    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b13f5d06-aebb-4761-b06c-95ad8d82d77c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b13f5d06-aebb-4761-b06c-95ad8d82d77c" (UID: "b13f5d06-aebb-4761-b06c-95ad8d82d77c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:14:47.196720 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.196706    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-dshm" (OuterVolumeSpecName: "dshm") pod "b13f5d06-aebb-4761-b06c-95ad8d82d77c" (UID: "b13f5d06-aebb-4761-b06c-95ad8d82d77c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:14:47.196767 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.196746    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b13f5d06-aebb-4761-b06c-95ad8d82d77c-kube-api-access-n6z8g" (OuterVolumeSpecName: "kube-api-access-n6z8g") pod "b13f5d06-aebb-4761-b06c-95ad8d82d77c" (UID: "b13f5d06-aebb-4761-b06c-95ad8d82d77c"). InnerVolumeSpecName "kube-api-access-n6z8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:14:47.295333 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.295282    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-model-cache\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:14:47.295333 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.295321    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b13f5d06-aebb-4761-b06c-95ad8d82d77c-tls-certs\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:14:47.295333 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.295331    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6z8g\" (UniqueName: \"kubernetes.io/projected/b13f5d06-aebb-4761-b06c-95ad8d82d77c-kube-api-access-n6z8g\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:14:47.295333 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:47.295341    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b13f5d06-aebb-4761-b06c-95ad8d82d77c-dshm\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:14:48.062182 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:48.062152    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"
Jul 06 18:14:48.062606 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:48.062188    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99" event={"ID":"b13f5d06-aebb-4761-b06c-95ad8d82d77c","Type":"ContainerDied","Data":"dfa89a31c199244d6b1b1c67f39fa11af6a7766f7362c358027f227fbbc1dcfa"}
Jul 06 18:14:48.062606 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:48.062242    2581 scope.go:117] "RemoveContainer" containerID="36d7c18f0e0d97b38793b181ae73204b68ec1946fedc6d08ae49cb45544a0971"
Jul 06 18:14:48.083012 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:48.082976    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"]
Jul 06 18:14:48.086710 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:48.086681    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-14d0a8da/e2e-lifecycle-model-14d0a8da-kserve-96c48f895-2pt99"]
Jul 06 18:14:48.912812 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:14:48.912782    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b13f5d06-aebb-4761-b06c-95ad8d82d77c" path="/var/lib/kubelet/pods/b13f5d06-aebb-4761-b06c-95ad8d82d77c/volumes"
Jul 06 18:15:00.134434 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:00.134398    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722695-47mn6"]
Jul 06 18:15:00.134936 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:00.134764    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b13f5d06-aebb-4761-b06c-95ad8d82d77c" containerName="main"
Jul 06 18:15:00.134936 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:00.134777    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b13f5d06-aebb-4761-b06c-95ad8d82d77c" containerName="main"
Jul 06 18:15:00.134936 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:00.134794    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="384971a1-d1d0-4b28-8ab3-4ee328fddcea" containerName="maas-api"
Jul 06 18:15:00.134936 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:00.134799    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="384971a1-d1d0-4b28-8ab3-4ee328fddcea" containerName="maas-api"
Jul 06 18:15:00.134936 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:00.134861    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="384971a1-d1d0-4b28-8ab3-4ee328fddcea" containerName="maas-api"
Jul 06 18:15:00.134936 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:00.134869    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b13f5d06-aebb-4761-b06c-95ad8d82d77c" containerName="main"
Jul 06 18:15:00.137780 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:00.137761    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722695-47mn6"
Jul 06 18:15:00.145702 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:00.145669    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722695-47mn6"]
Jul 06 18:15:00.200920 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:00.200855    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c7wt\" (UniqueName: \"kubernetes.io/projected/46d55e05-a713-48ea-966c-c7210d1ff7c6-kube-api-access-2c7wt\") pod \"maas-api-key-cleanup-29722695-47mn6\" (UID: \"46d55e05-a713-48ea-966c-c7210d1ff7c6\") " pod="opendatahub/maas-api-key-cleanup-29722695-47mn6"
Jul 06 18:15:00.301405 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:00.301363    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c7wt\" (UniqueName: \"kubernetes.io/projected/46d55e05-a713-48ea-966c-c7210d1ff7c6-kube-api-access-2c7wt\") pod \"maas-api-key-cleanup-29722695-47mn6\" (UID: \"46d55e05-a713-48ea-966c-c7210d1ff7c6\") " pod="opendatahub/maas-api-key-cleanup-29722695-47mn6"
Jul 06 18:15:00.309865 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:00.309834    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c7wt\" (UniqueName: \"kubernetes.io/projected/46d55e05-a713-48ea-966c-c7210d1ff7c6-kube-api-access-2c7wt\") pod \"maas-api-key-cleanup-29722695-47mn6\" (UID: \"46d55e05-a713-48ea-966c-c7210d1ff7c6\") " pod="opendatahub/maas-api-key-cleanup-29722695-47mn6"
Jul 06 18:15:00.449543 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:00.449498    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722695-47mn6"
Jul 06 18:15:00.593379 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:00.593197    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722695-47mn6"]
Jul 06 18:15:00.596787 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:15:00.596744    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46d55e05_a713_48ea_966c_c7210d1ff7c6.slice/crio-fe3bda7888fd8a53870c084af5598a7b1ca4520356ab74dc3aa31e4a002f623c WatchSource:0}: Error finding container fe3bda7888fd8a53870c084af5598a7b1ca4520356ab74dc3aa31e4a002f623c: Status 404 returned error can't find the container with id fe3bda7888fd8a53870c084af5598a7b1ca4520356ab74dc3aa31e4a002f623c
Jul 06 18:15:01.135025 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:01.134922    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722695-47mn6" event={"ID":"46d55e05-a713-48ea-966c-c7210d1ff7c6","Type":"ContainerStarted","Data":"65d521511f83b3145bd59f4b210f5c989b06e3347f5788325421547b14adddd8"}
Jul 06 18:15:01.135025 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:01.134978    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722695-47mn6" event={"ID":"46d55e05-a713-48ea-966c-c7210d1ff7c6","Type":"ContainerStarted","Data":"fe3bda7888fd8a53870c084af5598a7b1ca4520356ab74dc3aa31e4a002f623c"}
Jul 06 18:15:01.150763 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:01.150705    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722695-47mn6" podStartSLOduration=1.150685424 podStartE2EDuration="1.150685424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:15:01.148788372 +0000 UTC m=+2076.742242913" watchObservedRunningTime="2026-07-06 18:15:01.150685424 +0000 UTC m=+2076.744139953"
Jul 06 18:15:02.141507 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:02.141462    2581 generic.go:356] "Generic (PLEG): container finished" podID="46d55e05-a713-48ea-966c-c7210d1ff7c6" containerID="65d521511f83b3145bd59f4b210f5c989b06e3347f5788325421547b14adddd8" exitCode=0
Jul 06 18:15:02.141996 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:02.141523    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722695-47mn6" event={"ID":"46d55e05-a713-48ea-966c-c7210d1ff7c6","Type":"ContainerDied","Data":"65d521511f83b3145bd59f4b210f5c989b06e3347f5788325421547b14adddd8"}
Jul 06 18:15:03.290068 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:03.290036    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722695-47mn6"
Jul 06 18:15:03.328128 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:03.328085    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2c7wt\" (UniqueName: \"kubernetes.io/projected/46d55e05-a713-48ea-966c-c7210d1ff7c6-kube-api-access-2c7wt\") pod \"46d55e05-a713-48ea-966c-c7210d1ff7c6\" (UID: \"46d55e05-a713-48ea-966c-c7210d1ff7c6\") "
Jul 06 18:15:03.330217 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:03.330172    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46d55e05-a713-48ea-966c-c7210d1ff7c6-kube-api-access-2c7wt" (OuterVolumeSpecName: "kube-api-access-2c7wt") pod "46d55e05-a713-48ea-966c-c7210d1ff7c6" (UID: "46d55e05-a713-48ea-966c-c7210d1ff7c6"). InnerVolumeSpecName "kube-api-access-2c7wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:15:03.429145 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:03.429029    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c7wt\" (UniqueName: \"kubernetes.io/projected/46d55e05-a713-48ea-966c-c7210d1ff7c6-kube-api-access-2c7wt\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:15:04.155277 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:04.155227    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722695-47mn6"
Jul 06 18:15:04.155489 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:04.155217    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722695-47mn6" event={"ID":"46d55e05-a713-48ea-966c-c7210d1ff7c6","Type":"ContainerDied","Data":"fe3bda7888fd8a53870c084af5598a7b1ca4520356ab74dc3aa31e4a002f623c"}
Jul 06 18:15:04.155591 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:15:04.155568    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe3bda7888fd8a53870c084af5598a7b1ca4520356ab74dc3aa31e4a002f623c"
Jul 06 18:16:16.516378 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.516273    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"]
Jul 06 18:16:16.517039 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.516725    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46d55e05-a713-48ea-966c-c7210d1ff7c6" containerName="cleanup"
Jul 06 18:16:16.517039 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.516741    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="46d55e05-a713-48ea-966c-c7210d1ff7c6" containerName="cleanup"
Jul 06 18:16:16.517039 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.516812    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="46d55e05-a713-48ea-966c-c7210d1ff7c6" containerName="cleanup"
Jul 06 18:16:16.519807 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.519774    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:16:16.522602 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.522576    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-205c4c-serving-cert\""
Jul 06 18:16:16.530419 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.530374    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"]
Jul 06 18:16:16.651095 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.651055    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/687a17ca-0e05-4a71-8d22-309683e654ea-openshift-service-ca\") pod \"maas-api-e2e-inf-a-205c4c-786745767f-7wp85\" (UID: \"687a17ca-0e05-4a71-8d22-309683e654ea\") " pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:16:16.651284 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.651113    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/687a17ca-0e05-4a71-8d22-309683e654ea-maas-api-tls\") pod \"maas-api-e2e-inf-a-205c4c-786745767f-7wp85\" (UID: \"687a17ca-0e05-4a71-8d22-309683e654ea\") " pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:16:16.651284 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.651179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs6dt\" (UniqueName: \"kubernetes.io/projected/687a17ca-0e05-4a71-8d22-309683e654ea-kube-api-access-xs6dt\") pod \"maas-api-e2e-inf-a-205c4c-786745767f-7wp85\" (UID: \"687a17ca-0e05-4a71-8d22-309683e654ea\") " pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:16:16.751798 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.751755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/687a17ca-0e05-4a71-8d22-309683e654ea-openshift-service-ca\") pod \"maas-api-e2e-inf-a-205c4c-786745767f-7wp85\" (UID: \"687a17ca-0e05-4a71-8d22-309683e654ea\") " pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:16:16.752006 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.751809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/687a17ca-0e05-4a71-8d22-309683e654ea-maas-api-tls\") pod \"maas-api-e2e-inf-a-205c4c-786745767f-7wp85\" (UID: \"687a17ca-0e05-4a71-8d22-309683e654ea\") " pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:16:16.752006 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.751849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs6dt\" (UniqueName: \"kubernetes.io/projected/687a17ca-0e05-4a71-8d22-309683e654ea-kube-api-access-xs6dt\") pod \"maas-api-e2e-inf-a-205c4c-786745767f-7wp85\" (UID: \"687a17ca-0e05-4a71-8d22-309683e654ea\") " pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:16:16.752602 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.752571    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/687a17ca-0e05-4a71-8d22-309683e654ea-openshift-service-ca\") pod \"maas-api-e2e-inf-a-205c4c-786745767f-7wp85\" (UID: \"687a17ca-0e05-4a71-8d22-309683e654ea\") " pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:16:16.754420 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.754394    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/687a17ca-0e05-4a71-8d22-309683e654ea-maas-api-tls\") pod \"maas-api-e2e-inf-a-205c4c-786745767f-7wp85\" (UID: \"687a17ca-0e05-4a71-8d22-309683e654ea\") " pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:16:16.760493 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.760462    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs6dt\" (UniqueName: \"kubernetes.io/projected/687a17ca-0e05-4a71-8d22-309683e654ea-kube-api-access-xs6dt\") pod \"maas-api-e2e-inf-a-205c4c-786745767f-7wp85\" (UID: \"687a17ca-0e05-4a71-8d22-309683e654ea\") " pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:16:16.833361 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.833257    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:16:16.982781 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.982752    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"]
Jul 06 18:16:16.985642 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:16:16.985605    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod687a17ca_0e05_4a71_8d22_309683e654ea.slice/crio-f96575e05f983afcad3a29dbb074682de42b66a97c2cdf118b57dbe435e0ffdd WatchSource:0}: Error finding container f96575e05f983afcad3a29dbb074682de42b66a97c2cdf118b57dbe435e0ffdd: Status 404 returned error can't find the container with id f96575e05f983afcad3a29dbb074682de42b66a97c2cdf118b57dbe435e0ffdd
Jul 06 18:16:16.987204 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:16.987183    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 18:16:17.523607 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:17.523571    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85" event={"ID":"687a17ca-0e05-4a71-8d22-309683e654ea","Type":"ContainerStarted","Data":"f96575e05f983afcad3a29dbb074682de42b66a97c2cdf118b57dbe435e0ffdd"}
Jul 06 18:16:18.529588 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:18.529551    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85" event={"ID":"687a17ca-0e05-4a71-8d22-309683e654ea","Type":"ContainerStarted","Data":"9ea8784b4701dd5bf1fcaf8850c61fd523a4d9251a666082fdd0286babccbd39"}
Jul 06 18:16:18.530094 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:18.529707    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:16:18.547061 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:18.547002    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85" podStartSLOduration=1.9831965459999998 podStartE2EDuration="2.546984396s" totalImagesPullingTime="563.78785ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:16:18.545179857 +0000 UTC m=+2154.138634386" watchObservedRunningTime="2026-07-06 18:16:18.546984396 +0000 UTC m=+2154.140438925"
Jul 06 18:16:24.540460 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.540424    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:16:24.740012 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.739969    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"]
Jul 06 18:16:24.744025 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.743998    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:16:24.747508 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.747474    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-c34ce0-serving-cert\""
Jul 06 18:16:24.767156 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.767112    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"]
Jul 06 18:16:24.829384 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.829272    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d54e5613-7420-4491-bc8a-a8db482d9cd3-maas-api-tls\") pod \"maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72\" (UID: \"d54e5613-7420-4491-bc8a-a8db482d9cd3\") " pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:16:24.829384 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.829317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk2nt\" (UniqueName: \"kubernetes.io/projected/d54e5613-7420-4491-bc8a-a8db482d9cd3-kube-api-access-bk2nt\") pod \"maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72\" (UID: \"d54e5613-7420-4491-bc8a-a8db482d9cd3\") " pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:16:24.829384 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.829356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d54e5613-7420-4491-bc8a-a8db482d9cd3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72\" (UID: \"d54e5613-7420-4491-bc8a-a8db482d9cd3\") " pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:16:24.930586 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.930543    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d54e5613-7420-4491-bc8a-a8db482d9cd3-maas-api-tls\") pod \"maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72\" (UID: \"d54e5613-7420-4491-bc8a-a8db482d9cd3\") " pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:16:24.930586 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.930589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk2nt\" (UniqueName: \"kubernetes.io/projected/d54e5613-7420-4491-bc8a-a8db482d9cd3-kube-api-access-bk2nt\") pod \"maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72\" (UID: \"d54e5613-7420-4491-bc8a-a8db482d9cd3\") " pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:16:24.930831 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.930636    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d54e5613-7420-4491-bc8a-a8db482d9cd3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72\" (UID: \"d54e5613-7420-4491-bc8a-a8db482d9cd3\") " pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:16:24.931386 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.931366    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d54e5613-7420-4491-bc8a-a8db482d9cd3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72\" (UID: \"d54e5613-7420-4491-bc8a-a8db482d9cd3\") " pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:16:24.933381 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.933361    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-c34ce0-serving-cert\""
Jul 06 18:16:24.939441 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.939412    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk2nt\" (UniqueName: \"kubernetes.io/projected/d54e5613-7420-4491-bc8a-a8db482d9cd3-kube-api-access-bk2nt\") pod \"maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72\" (UID: \"d54e5613-7420-4491-bc8a-a8db482d9cd3\") " pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:16:24.943403 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:24.943371    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d54e5613-7420-4491-bc8a-a8db482d9cd3-maas-api-tls\") pod \"maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72\" (UID: \"d54e5613-7420-4491-bc8a-a8db482d9cd3\") " pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:16:25.056500 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:25.056434    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:16:25.206407 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:25.206371    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"]
Jul 06 18:16:25.207968 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:16:25.207942    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd54e5613_7420_4491_bc8a_a8db482d9cd3.slice/crio-c1f3f3542ac3068571694803dc8468edf12bd2df88ba574c262f67b01c84af07 WatchSource:0}: Error finding container c1f3f3542ac3068571694803dc8468edf12bd2df88ba574c262f67b01c84af07: Status 404 returned error can't find the container with id c1f3f3542ac3068571694803dc8468edf12bd2df88ba574c262f67b01c84af07
Jul 06 18:16:25.568013 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:25.567978    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72" event={"ID":"d54e5613-7420-4491-bc8a-a8db482d9cd3","Type":"ContainerStarted","Data":"c1f3f3542ac3068571694803dc8468edf12bd2df88ba574c262f67b01c84af07"}
Jul 06 18:16:26.575043 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:26.574880    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72" event={"ID":"d54e5613-7420-4491-bc8a-a8db482d9cd3","Type":"ContainerStarted","Data":"f363e22951eca10b7f7633039403cb786c8e027f097c2d403d419ce9d4e748fb"}
Jul 06 18:16:26.575043 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:26.574975    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:16:26.592631 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:26.592573    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72" podStartSLOduration=2.235104557 podStartE2EDuration="2.5925543s" totalImagesPullingTime="357.449743ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:16:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:16:26.590979649 +0000 UTC m=+2162.184434181" watchObservedRunningTime="2026-07-06 18:16:26.5925543 +0000 UTC m=+2162.186008829"
Jul 06 18:16:32.585884 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:32.585854    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:16:41.099824 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.099778    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"]
Jul 06 18:16:41.103561 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.103534    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.106308 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.106280    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-205c4c\"/\"kube-root-ca.crt\""
Jul 06 18:16:41.107620 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.107598    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-205c4c\"/\"test-model-205c4c-kserve-self-signed-certs\""
Jul 06 18:16:41.107756 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.107699    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-205c4c\"/\"openshift-service-ca.crt\""
Jul 06 18:16:41.107756 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.107710    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-205c4c\"/\"default-dockercfg-hbf4r\""
Jul 06 18:16:41.112148 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.112117    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"]
Jul 06 18:16:41.161985 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.161948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-home\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.162183 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.162039    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-tls-certs\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.162183 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.162089    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-tmp-dir\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.162183 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.162161    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-model-cache\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.162330 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.162220    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qq8t\" (UniqueName: \"kubernetes.io/projected/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-kube-api-access-2qq8t\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.162330 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.162282    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-dshm\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.262620 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.262579    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-tmp-dir\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.262620 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.262623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-model-cache\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.262862 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.262644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qq8t\" (UniqueName: \"kubernetes.io/projected/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-kube-api-access-2qq8t\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.262862 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.262775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-dshm\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.262988 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.262876    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-home\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.262988 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.262963    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-tls-certs\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.263097 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.263078    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-tmp-dir\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.263132 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.263118    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-model-cache\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.263253 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.263237    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-home\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.265166 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.265142    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-dshm\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.265450 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.265424    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-tls-certs\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.271809 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.271776    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qq8t\" (UniqueName: \"kubernetes.io/projected/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-kube-api-access-2qq8t\") pod \"test-model-205c4c-kserve-746f98b9bc-vqnc2\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") " pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.414540 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.414434    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:41.560181 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.560148    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"]
Jul 06 18:16:41.561914 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:16:41.561868    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52791fa0_69b5_4b46_b33b_80b0bf7e39ce.slice/crio-38ecd1b967d20ebbf6c4d0f9f80a45c79eb4617d19707a7c4c2ed5a03dccd732 WatchSource:0}: Error finding container 38ecd1b967d20ebbf6c4d0f9f80a45c79eb4617d19707a7c4c2ed5a03dccd732: Status 404 returned error can't find the container with id 38ecd1b967d20ebbf6c4d0f9f80a45c79eb4617d19707a7c4c2ed5a03dccd732
Jul 06 18:16:41.653855 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.653814    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2" event={"ID":"52791fa0-69b5-4b46-b33b-80b0bf7e39ce","Type":"ContainerStarted","Data":"81cd7f7437b46d38364ac26abed45a8c99c3b17464cb8b9aaf24aef094decf34"}
Jul 06 18:16:41.654091 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.653869    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2" event={"ID":"52791fa0-69b5-4b46-b33b-80b0bf7e39ce","Type":"ContainerStarted","Data":"38ecd1b967d20ebbf6c4d0f9f80a45c79eb4617d19707a7c4c2ed5a03dccd732"}
Jul 06 18:16:41.670246 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:41.670134    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2" podStartSLOduration=0.670116064 podStartE2EDuration="670.116064ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:16:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:16:41.669058997 +0000 UTC m=+2177.262513520" watchObservedRunningTime="2026-07-06 18:16:41.670116064 +0000 UTC m=+2177.263570629"
Jul 06 18:16:51.414720 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:51.414675    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:51.415351 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:51.414771    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:51.429917 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:51.429864    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:16:51.721758 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:16:51.721723    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:17:02.391584 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.391545    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"]
Jul 06 18:17:02.395431 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.395402    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.399167 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.399144    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c34ce0\"/\"default-dockercfg-tj8nn\""
Jul 06 18:17:02.399167 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.399160    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c34ce0\"/\"kube-root-ca.crt\""
Jul 06 18:17:02.399366 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.399158    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c34ce0\"/\"openshift-service-ca.crt\""
Jul 06 18:17:02.399366 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.399202    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c34ce0\"/\"test-model-c34ce0-kserve-self-signed-certs\""
Jul 06 18:17:02.402388 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.402324    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"]
Jul 06 18:17:02.433540 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.433493    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b439d91d-077e-4729-a960-2f73882fbfde-tls-certs\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.433540 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.433537    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8tq8\" (UniqueName: \"kubernetes.io/projected/b439d91d-077e-4729-a960-2f73882fbfde-kube-api-access-g8tq8\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.433845 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.433631    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-dshm\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.433845 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.433648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-tmp-dir\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.433845 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.433668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-model-cache\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.433845 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.433684    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-home\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.534728 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.534679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b439d91d-077e-4729-a960-2f73882fbfde-tls-certs\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.534728 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.534725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8tq8\" (UniqueName: \"kubernetes.io/projected/b439d91d-077e-4729-a960-2f73882fbfde-kube-api-access-g8tq8\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.535007 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.534825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-dshm\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.535007 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.534854    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-tmp-dir\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.535007 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.534885    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-model-cache\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.535007 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.534947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-home\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.535383 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.535354    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-tmp-dir\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.535498 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.535413    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-model-cache\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.535498 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.535447    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-home\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.537211 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.537186    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-dshm\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.537490 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.537470    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b439d91d-077e-4729-a960-2f73882fbfde-tls-certs\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.543323 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.543283    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8tq8\" (UniqueName: \"kubernetes.io/projected/b439d91d-077e-4729-a960-2f73882fbfde-kube-api-access-g8tq8\") pod \"test-model-c34ce0-kserve-78764854f7-gx4dn\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") " pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.708059 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.708011    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:02.847496 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:02.847461    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"]
Jul 06 18:17:02.849355 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:17:02.849323    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb439d91d_077e_4729_a960_2f73882fbfde.slice/crio-a3469a6b132f4a230ec0f2b144d45c7dd483388f8fbdb4aa0054dbbef438fa81 WatchSource:0}: Error finding container a3469a6b132f4a230ec0f2b144d45c7dd483388f8fbdb4aa0054dbbef438fa81: Status 404 returned error can't find the container with id a3469a6b132f4a230ec0f2b144d45c7dd483388f8fbdb4aa0054dbbef438fa81
Jul 06 18:17:03.773972 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:03.773930    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn" event={"ID":"b439d91d-077e-4729-a960-2f73882fbfde","Type":"ContainerStarted","Data":"b04c3dda75df906c4c119eafdd927fef8b5b3d20fde4e1b6657a2d06057a83a2"}
Jul 06 18:17:03.773972 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:03.773980    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn" event={"ID":"b439d91d-077e-4729-a960-2f73882fbfde","Type":"ContainerStarted","Data":"a3469a6b132f4a230ec0f2b144d45c7dd483388f8fbdb4aa0054dbbef438fa81"}
Jul 06 18:17:03.790229 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:03.790166    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn" podStartSLOduration=1.7901495779999999 podStartE2EDuration="1.790149578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:17:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:17:03.788038926 +0000 UTC m=+2199.381493496" watchObservedRunningTime="2026-07-06 18:17:03.790149578 +0000 UTC m=+2199.383604108"
Jul 06 18:17:12.708581 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:12.708514    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:12.708581 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:12.708596    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:12.721281 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:12.721252    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:12.832687 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:12.832660    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:22.476931 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:22.476878    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"]
Jul 06 18:17:23.472408 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:23.472366    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"]
Jul 06 18:17:24.260403 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.260327    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"]
Jul 06 18:17:24.260965 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.260578    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85" podUID="687a17ca-0e05-4a71-8d22-309683e654ea" containerName="maas-api" containerID="cri-o://9ea8784b4701dd5bf1fcaf8850c61fd523a4d9251a666082fdd0286babccbd39" gracePeriod=30
Jul 06 18:17:24.535247 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.535203    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85" podUID="687a17ca-0e05-4a71-8d22-309683e654ea" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.54:8443/health\": dial tcp 10.132.0.54:8443: connect: connection refused"
Jul 06 18:17:24.726233 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.726201    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:17:24.851037 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.850947    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/687a17ca-0e05-4a71-8d22-309683e654ea-openshift-service-ca\") pod \"687a17ca-0e05-4a71-8d22-309683e654ea\" (UID: \"687a17ca-0e05-4a71-8d22-309683e654ea\") "
Jul 06 18:17:24.851195 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.851061    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/687a17ca-0e05-4a71-8d22-309683e654ea-maas-api-tls\") pod \"687a17ca-0e05-4a71-8d22-309683e654ea\" (UID: \"687a17ca-0e05-4a71-8d22-309683e654ea\") "
Jul 06 18:17:24.851195 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.851079    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xs6dt\" (UniqueName: \"kubernetes.io/projected/687a17ca-0e05-4a71-8d22-309683e654ea-kube-api-access-xs6dt\") pod \"687a17ca-0e05-4a71-8d22-309683e654ea\" (UID: \"687a17ca-0e05-4a71-8d22-309683e654ea\") "
Jul 06 18:17:24.851368 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.851343    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/687a17ca-0e05-4a71-8d22-309683e654ea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "687a17ca-0e05-4a71-8d22-309683e654ea" (UID: "687a17ca-0e05-4a71-8d22-309683e654ea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:17:24.853255 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.853210    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/687a17ca-0e05-4a71-8d22-309683e654ea-kube-api-access-xs6dt" (OuterVolumeSpecName: "kube-api-access-xs6dt") pod "687a17ca-0e05-4a71-8d22-309683e654ea" (UID: "687a17ca-0e05-4a71-8d22-309683e654ea"). InnerVolumeSpecName "kube-api-access-xs6dt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:17:24.853255 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.853230    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/687a17ca-0e05-4a71-8d22-309683e654ea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "687a17ca-0e05-4a71-8d22-309683e654ea" (UID: "687a17ca-0e05-4a71-8d22-309683e654ea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:17:24.885441 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.885409    2581 generic.go:356] "Generic (PLEG): container finished" podID="687a17ca-0e05-4a71-8d22-309683e654ea" containerID="9ea8784b4701dd5bf1fcaf8850c61fd523a4d9251a666082fdd0286babccbd39" exitCode=0
Jul 06 18:17:24.885621 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.885456    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"
Jul 06 18:17:24.885621 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.885454    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85" event={"ID":"687a17ca-0e05-4a71-8d22-309683e654ea","Type":"ContainerDied","Data":"9ea8784b4701dd5bf1fcaf8850c61fd523a4d9251a666082fdd0286babccbd39"}
Jul 06 18:17:24.885621 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.885587    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85" event={"ID":"687a17ca-0e05-4a71-8d22-309683e654ea","Type":"ContainerDied","Data":"f96575e05f983afcad3a29dbb074682de42b66a97c2cdf118b57dbe435e0ffdd"}
Jul 06 18:17:24.885621 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.885615    2581 scope.go:117] "RemoveContainer" containerID="9ea8784b4701dd5bf1fcaf8850c61fd523a4d9251a666082fdd0286babccbd39"
Jul 06 18:17:24.895022 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.894999    2581 scope.go:117] "RemoveContainer" containerID="9ea8784b4701dd5bf1fcaf8850c61fd523a4d9251a666082fdd0286babccbd39"
Jul 06 18:17:24.895357 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:17:24.895336    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ea8784b4701dd5bf1fcaf8850c61fd523a4d9251a666082fdd0286babccbd39\": container with ID starting with 9ea8784b4701dd5bf1fcaf8850c61fd523a4d9251a666082fdd0286babccbd39 not found: ID does not exist" containerID="9ea8784b4701dd5bf1fcaf8850c61fd523a4d9251a666082fdd0286babccbd39"
Jul 06 18:17:24.895424 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.895367    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ea8784b4701dd5bf1fcaf8850c61fd523a4d9251a666082fdd0286babccbd39"} err="failed to get container status \"9ea8784b4701dd5bf1fcaf8850c61fd523a4d9251a666082fdd0286babccbd39\": rpc error: code = NotFound desc = could not find container \"9ea8784b4701dd5bf1fcaf8850c61fd523a4d9251a666082fdd0286babccbd39\": container with ID starting with 9ea8784b4701dd5bf1fcaf8850c61fd523a4d9251a666082fdd0286babccbd39 not found: ID does not exist"
Jul 06 18:17:24.913161 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.913130    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"]
Jul 06 18:17:24.913161 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.913169    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-205c4c-786745767f-7wp85"]
Jul 06 18:17:24.952279 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.952242    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/687a17ca-0e05-4a71-8d22-309683e654ea-maas-api-tls\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:24.952279 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.952271    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xs6dt\" (UniqueName: \"kubernetes.io/projected/687a17ca-0e05-4a71-8d22-309683e654ea-kube-api-access-xs6dt\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:24.952279 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:24.952282    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/687a17ca-0e05-4a71-8d22-309683e654ea-openshift-service-ca\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:26.913119 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:26.913082    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="687a17ca-0e05-4a71-8d22-309683e654ea" path="/var/lib/kubelet/pods/687a17ca-0e05-4a71-8d22-309683e654ea/volumes"
Jul 06 18:17:37.498906 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.498843    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2" podUID="52791fa0-69b5-4b46-b33b-80b0bf7e39ce" containerName="main" containerID="cri-o://81cd7f7437b46d38364ac26abed45a8c99c3b17464cb8b9aaf24aef094decf34" gracePeriod=45
Jul 06 18:17:37.771324 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.771292    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:17:37.868001 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.867967    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-model-cache\") pod \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") "
Jul 06 18:17:37.868001 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.868006    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-dshm\") pod \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") "
Jul 06 18:17:37.868263 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.868036    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-home\") pod \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") "
Jul 06 18:17:37.868263 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.868079    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-tmp-dir\") pod \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") "
Jul 06 18:17:37.868263 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.868106    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-tls-certs\") pod \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") "
Jul 06 18:17:37.868263 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.868150    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qq8t\" (UniqueName: \"kubernetes.io/projected/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-kube-api-access-2qq8t\") pod \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\" (UID: \"52791fa0-69b5-4b46-b33b-80b0bf7e39ce\") "
Jul 06 18:17:37.868485 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.868321    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "52791fa0-69b5-4b46-b33b-80b0bf7e39ce" (UID: "52791fa0-69b5-4b46-b33b-80b0bf7e39ce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:17:37.868485 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.868333    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-home" (OuterVolumeSpecName: "home") pod "52791fa0-69b5-4b46-b33b-80b0bf7e39ce" (UID: "52791fa0-69b5-4b46-b33b-80b0bf7e39ce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:17:37.868485 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.868340    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-model-cache" (OuterVolumeSpecName: "model-cache") pod "52791fa0-69b5-4b46-b33b-80b0bf7e39ce" (UID: "52791fa0-69b5-4b46-b33b-80b0bf7e39ce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:17:37.868485 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.868433    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-tmp-dir\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:37.868485 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.868455    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-model-cache\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:37.868485 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.868469    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-home\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:37.870571 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.870530    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-kube-api-access-2qq8t" (OuterVolumeSpecName: "kube-api-access-2qq8t") pod "52791fa0-69b5-4b46-b33b-80b0bf7e39ce" (UID: "52791fa0-69b5-4b46-b33b-80b0bf7e39ce"). InnerVolumeSpecName "kube-api-access-2qq8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:17:37.870571 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.870563    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-dshm" (OuterVolumeSpecName: "dshm") pod "52791fa0-69b5-4b46-b33b-80b0bf7e39ce" (UID: "52791fa0-69b5-4b46-b33b-80b0bf7e39ce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:17:37.870758 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.870601    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "52791fa0-69b5-4b46-b33b-80b0bf7e39ce" (UID: "52791fa0-69b5-4b46-b33b-80b0bf7e39ce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:17:37.961222 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.961189    2581 generic.go:356] "Generic (PLEG): container finished" podID="52791fa0-69b5-4b46-b33b-80b0bf7e39ce" containerID="81cd7f7437b46d38364ac26abed45a8c99c3b17464cb8b9aaf24aef094decf34" exitCode=0
Jul 06 18:17:37.961377 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.961235    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"
Jul 06 18:17:37.961377 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.961274    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2" event={"ID":"52791fa0-69b5-4b46-b33b-80b0bf7e39ce","Type":"ContainerDied","Data":"81cd7f7437b46d38364ac26abed45a8c99c3b17464cb8b9aaf24aef094decf34"}
Jul 06 18:17:37.961377 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.961320    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2" event={"ID":"52791fa0-69b5-4b46-b33b-80b0bf7e39ce","Type":"ContainerDied","Data":"38ecd1b967d20ebbf6c4d0f9f80a45c79eb4617d19707a7c4c2ed5a03dccd732"}
Jul 06 18:17:37.961377 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.961338    2581 scope.go:117] "RemoveContainer" containerID="81cd7f7437b46d38364ac26abed45a8c99c3b17464cb8b9aaf24aef094decf34"
Jul 06 18:17:37.969255 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.969219    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-tls-certs\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:37.969255 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.969250    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qq8t\" (UniqueName: \"kubernetes.io/projected/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-kube-api-access-2qq8t\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:37.969464 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.969264    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52791fa0-69b5-4b46-b33b-80b0bf7e39ce-dshm\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:37.971316 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.971297    2581 scope.go:117] "RemoveContainer" containerID="81cd7f7437b46d38364ac26abed45a8c99c3b17464cb8b9aaf24aef094decf34"
Jul 06 18:17:37.971614 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:17:37.971593    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81cd7f7437b46d38364ac26abed45a8c99c3b17464cb8b9aaf24aef094decf34\": container with ID starting with 81cd7f7437b46d38364ac26abed45a8c99c3b17464cb8b9aaf24aef094decf34 not found: ID does not exist" containerID="81cd7f7437b46d38364ac26abed45a8c99c3b17464cb8b9aaf24aef094decf34"
Jul 06 18:17:37.971682 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.971625    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81cd7f7437b46d38364ac26abed45a8c99c3b17464cb8b9aaf24aef094decf34"} err="failed to get container status \"81cd7f7437b46d38364ac26abed45a8c99c3b17464cb8b9aaf24aef094decf34\": rpc error: code = NotFound desc = could not find container \"81cd7f7437b46d38364ac26abed45a8c99c3b17464cb8b9aaf24aef094decf34\": container with ID starting with 81cd7f7437b46d38364ac26abed45a8c99c3b17464cb8b9aaf24aef094decf34 not found: ID does not exist"
Jul 06 18:17:37.983598 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.983558    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"]
Jul 06 18:17:37.987191 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:37.987160    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-205c4c/test-model-205c4c-kserve-746f98b9bc-vqnc2"]
Jul 06 18:17:38.494147 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.494104    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn" podUID="b439d91d-077e-4729-a960-2f73882fbfde" containerName="main" containerID="cri-o://b04c3dda75df906c4c119eafdd927fef8b5b3d20fde4e1b6657a2d06057a83a2" gracePeriod=45
Jul 06 18:17:38.755867 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.755793    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:38.776234 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.776199    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-home\") pod \"b439d91d-077e-4729-a960-2f73882fbfde\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") "
Jul 06 18:17:38.776234 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.776238    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-model-cache\") pod \"b439d91d-077e-4729-a960-2f73882fbfde\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") "
Jul 06 18:17:38.776498 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.776295    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8tq8\" (UniqueName: \"kubernetes.io/projected/b439d91d-077e-4729-a960-2f73882fbfde-kube-api-access-g8tq8\") pod \"b439d91d-077e-4729-a960-2f73882fbfde\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") "
Jul 06 18:17:38.776498 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.776327    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b439d91d-077e-4729-a960-2f73882fbfde-tls-certs\") pod \"b439d91d-077e-4729-a960-2f73882fbfde\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") "
Jul 06 18:17:38.776498 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.776359    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-dshm\") pod \"b439d91d-077e-4729-a960-2f73882fbfde\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") "
Jul 06 18:17:38.776498 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.776397    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-tmp-dir\") pod \"b439d91d-077e-4729-a960-2f73882fbfde\" (UID: \"b439d91d-077e-4729-a960-2f73882fbfde\") "
Jul 06 18:17:38.776720 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.776529    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-home" (OuterVolumeSpecName: "home") pod "b439d91d-077e-4729-a960-2f73882fbfde" (UID: "b439d91d-077e-4729-a960-2f73882fbfde"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:17:38.776720 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.776626    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-home\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:38.776720 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.776672    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-model-cache" (OuterVolumeSpecName: "model-cache") pod "b439d91d-077e-4729-a960-2f73882fbfde" (UID: "b439d91d-077e-4729-a960-2f73882fbfde"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:17:38.776720 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.776703    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b439d91d-077e-4729-a960-2f73882fbfde" (UID: "b439d91d-077e-4729-a960-2f73882fbfde"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:17:38.779423 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.779388    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-dshm" (OuterVolumeSpecName: "dshm") pod "b439d91d-077e-4729-a960-2f73882fbfde" (UID: "b439d91d-077e-4729-a960-2f73882fbfde"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:17:38.779423 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.779407    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b439d91d-077e-4729-a960-2f73882fbfde-kube-api-access-g8tq8" (OuterVolumeSpecName: "kube-api-access-g8tq8") pod "b439d91d-077e-4729-a960-2f73882fbfde" (UID: "b439d91d-077e-4729-a960-2f73882fbfde"). InnerVolumeSpecName "kube-api-access-g8tq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:17:38.779423 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.779396    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b439d91d-077e-4729-a960-2f73882fbfde-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b439d91d-077e-4729-a960-2f73882fbfde" (UID: "b439d91d-077e-4729-a960-2f73882fbfde"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:17:38.877328 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.877294    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-model-cache\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:38.877328 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.877327    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8tq8\" (UniqueName: \"kubernetes.io/projected/b439d91d-077e-4729-a960-2f73882fbfde-kube-api-access-g8tq8\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:38.877559 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.877342    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b439d91d-077e-4729-a960-2f73882fbfde-tls-certs\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:38.877559 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.877355    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-dshm\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:38.877559 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.877368    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b439d91d-077e-4729-a960-2f73882fbfde-tmp-dir\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:17:38.912814 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.912770    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52791fa0-69b5-4b46-b33b-80b0bf7e39ce" path="/var/lib/kubelet/pods/52791fa0-69b5-4b46-b33b-80b0bf7e39ce/volumes"
Jul 06 18:17:38.967823 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.967791    2581 generic.go:356] "Generic (PLEG): container finished" podID="b439d91d-077e-4729-a960-2f73882fbfde" containerID="b04c3dda75df906c4c119eafdd927fef8b5b3d20fde4e1b6657a2d06057a83a2" exitCode=0
Jul 06 18:17:38.968037 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.967832    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"
Jul 06 18:17:38.968037 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.967842    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn" event={"ID":"b439d91d-077e-4729-a960-2f73882fbfde","Type":"ContainerDied","Data":"b04c3dda75df906c4c119eafdd927fef8b5b3d20fde4e1b6657a2d06057a83a2"}
Jul 06 18:17:38.968037 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.967880    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn" event={"ID":"b439d91d-077e-4729-a960-2f73882fbfde","Type":"ContainerDied","Data":"a3469a6b132f4a230ec0f2b144d45c7dd483388f8fbdb4aa0054dbbef438fa81"}
Jul 06 18:17:38.968037 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.967927    2581 scope.go:117] "RemoveContainer" containerID="b04c3dda75df906c4c119eafdd927fef8b5b3d20fde4e1b6657a2d06057a83a2"
Jul 06 18:17:38.977030 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.977007    2581 scope.go:117] "RemoveContainer" containerID="b04c3dda75df906c4c119eafdd927fef8b5b3d20fde4e1b6657a2d06057a83a2"
Jul 06 18:17:38.977355 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:17:38.977335    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b04c3dda75df906c4c119eafdd927fef8b5b3d20fde4e1b6657a2d06057a83a2\": container with ID starting with b04c3dda75df906c4c119eafdd927fef8b5b3d20fde4e1b6657a2d06057a83a2 not found: ID does not exist" containerID="b04c3dda75df906c4c119eafdd927fef8b5b3d20fde4e1b6657a2d06057a83a2"
Jul 06 18:17:38.977410 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.977365    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b04c3dda75df906c4c119eafdd927fef8b5b3d20fde4e1b6657a2d06057a83a2"} err="failed to get container status \"b04c3dda75df906c4c119eafdd927fef8b5b3d20fde4e1b6657a2d06057a83a2\": rpc error: code = NotFound desc = could not find container \"b04c3dda75df906c4c119eafdd927fef8b5b3d20fde4e1b6657a2d06057a83a2\": container with ID starting with b04c3dda75df906c4c119eafdd927fef8b5b3d20fde4e1b6657a2d06057a83a2 not found: ID does not exist"
Jul 06 18:17:38.984735 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.984699    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"]
Jul 06 18:17:38.990354 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:38.990304    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c34ce0/test-model-c34ce0-kserve-78764854f7-gx4dn"]
Jul 06 18:17:40.912673 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:17:40.912625    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b439d91d-077e-4729-a960-2f73882fbfde" path="/var/lib/kubelet/pods/b439d91d-077e-4729-a960-2f73882fbfde/volumes"
Jul 06 18:18:07.130886 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:07.130833    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"]
Jul 06 18:18:07.131510 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:07.131128    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72" podUID="d54e5613-7420-4491-bc8a-a8db482d9cd3" containerName="maas-api" containerID="cri-o://f363e22951eca10b7f7633039403cb786c8e027f097c2d403d419ce9d4e748fb" gracePeriod=30
Jul 06 18:18:07.591888 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:07.591843    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:18:07.720326 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:07.720283    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d54e5613-7420-4491-bc8a-a8db482d9cd3-maas-api-tls\") pod \"d54e5613-7420-4491-bc8a-a8db482d9cd3\" (UID: \"d54e5613-7420-4491-bc8a-a8db482d9cd3\") "
Jul 06 18:18:07.720326 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:07.720338    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d54e5613-7420-4491-bc8a-a8db482d9cd3-openshift-service-ca\") pod \"d54e5613-7420-4491-bc8a-a8db482d9cd3\" (UID: \"d54e5613-7420-4491-bc8a-a8db482d9cd3\") "
Jul 06 18:18:07.720611 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:07.720394    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bk2nt\" (UniqueName: \"kubernetes.io/projected/d54e5613-7420-4491-bc8a-a8db482d9cd3-kube-api-access-bk2nt\") pod \"d54e5613-7420-4491-bc8a-a8db482d9cd3\" (UID: \"d54e5613-7420-4491-bc8a-a8db482d9cd3\") "
Jul 06 18:18:07.720751 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:07.720722    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d54e5613-7420-4491-bc8a-a8db482d9cd3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d54e5613-7420-4491-bc8a-a8db482d9cd3" (UID: "d54e5613-7420-4491-bc8a-a8db482d9cd3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:18:07.722636 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:07.722604    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d54e5613-7420-4491-bc8a-a8db482d9cd3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d54e5613-7420-4491-bc8a-a8db482d9cd3" (UID: "d54e5613-7420-4491-bc8a-a8db482d9cd3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:18:07.722636 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:07.722623    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d54e5613-7420-4491-bc8a-a8db482d9cd3-kube-api-access-bk2nt" (OuterVolumeSpecName: "kube-api-access-bk2nt") pod "d54e5613-7420-4491-bc8a-a8db482d9cd3" (UID: "d54e5613-7420-4491-bc8a-a8db482d9cd3"). InnerVolumeSpecName "kube-api-access-bk2nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:18:07.820961 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:07.820918    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bk2nt\" (UniqueName: \"kubernetes.io/projected/d54e5613-7420-4491-bc8a-a8db482d9cd3-kube-api-access-bk2nt\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:18:07.820961 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:07.820950    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d54e5613-7420-4491-bc8a-a8db482d9cd3-maas-api-tls\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:18:07.820961 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:07.820959    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d54e5613-7420-4491-bc8a-a8db482d9cd3-openshift-service-ca\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:18:08.111106 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:08.111009    2581 generic.go:356] "Generic (PLEG): container finished" podID="d54e5613-7420-4491-bc8a-a8db482d9cd3" containerID="f363e22951eca10b7f7633039403cb786c8e027f097c2d403d419ce9d4e748fb" exitCode=0
Jul 06 18:18:08.111106 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:08.111056    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"
Jul 06 18:18:08.111106 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:08.111092    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72" event={"ID":"d54e5613-7420-4491-bc8a-a8db482d9cd3","Type":"ContainerDied","Data":"f363e22951eca10b7f7633039403cb786c8e027f097c2d403d419ce9d4e748fb"}
Jul 06 18:18:08.111365 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:08.111153    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72" event={"ID":"d54e5613-7420-4491-bc8a-a8db482d9cd3","Type":"ContainerDied","Data":"c1f3f3542ac3068571694803dc8468edf12bd2df88ba574c262f67b01c84af07"}
Jul 06 18:18:08.111365 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:08.111183    2581 scope.go:117] "RemoveContainer" containerID="f363e22951eca10b7f7633039403cb786c8e027f097c2d403d419ce9d4e748fb"
Jul 06 18:18:08.120450 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:08.120429    2581 scope.go:117] "RemoveContainer" containerID="f363e22951eca10b7f7633039403cb786c8e027f097c2d403d419ce9d4e748fb"
Jul 06 18:18:08.120776 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:18:08.120755    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f363e22951eca10b7f7633039403cb786c8e027f097c2d403d419ce9d4e748fb\": container with ID starting with f363e22951eca10b7f7633039403cb786c8e027f097c2d403d419ce9d4e748fb not found: ID does not exist" containerID="f363e22951eca10b7f7633039403cb786c8e027f097c2d403d419ce9d4e748fb"
Jul 06 18:18:08.120858 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:08.120784    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f363e22951eca10b7f7633039403cb786c8e027f097c2d403d419ce9d4e748fb"} err="failed to get container status \"f363e22951eca10b7f7633039403cb786c8e027f097c2d403d419ce9d4e748fb\": rpc error: code = NotFound desc = could not find container \"f363e22951eca10b7f7633039403cb786c8e027f097c2d403d419ce9d4e748fb\": container with ID starting with f363e22951eca10b7f7633039403cb786c8e027f097c2d403d419ce9d4e748fb not found: ID does not exist"
Jul 06 18:18:08.131511 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:08.131475    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"]
Jul 06 18:18:08.135707 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:08.135673    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72"]
Jul 06 18:18:08.913537 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:08.913490    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d54e5613-7420-4491-bc8a-a8db482d9cd3" path="/var/lib/kubelet/pods/d54e5613-7420-4491-bc8a-a8db482d9cd3/volumes"
Jul 06 18:18:10.580583 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:10.580525    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-b-c34ce0-6f89bbcd45-hzj72" podUID="d54e5613-7420-4491-bc8a-a8db482d9cd3" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.55:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 06 18:18:17.035039 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.034990    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"]
Jul 06 18:18:17.035563 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.035521    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="687a17ca-0e05-4a71-8d22-309683e654ea" containerName="maas-api"
Jul 06 18:18:17.035563 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.035542    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="687a17ca-0e05-4a71-8d22-309683e654ea" containerName="maas-api"
Jul 06 18:18:17.035687 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.035578    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d54e5613-7420-4491-bc8a-a8db482d9cd3" containerName="maas-api"
Jul 06 18:18:17.035687 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.035588    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d54e5613-7420-4491-bc8a-a8db482d9cd3" containerName="maas-api"
Jul 06 18:18:17.035687 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.035599    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b439d91d-077e-4729-a960-2f73882fbfde" containerName="main"
Jul 06 18:18:17.035687 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.035608    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b439d91d-077e-4729-a960-2f73882fbfde" containerName="main"
Jul 06 18:18:17.035687 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.035622    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52791fa0-69b5-4b46-b33b-80b0bf7e39ce" containerName="main"
Jul 06 18:18:17.035687 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.035630    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="52791fa0-69b5-4b46-b33b-80b0bf7e39ce" containerName="main"
Jul 06 18:18:17.036001 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.035725    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b439d91d-077e-4729-a960-2f73882fbfde" containerName="main"
Jul 06 18:18:17.036001 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.035740    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d54e5613-7420-4491-bc8a-a8db482d9cd3" containerName="maas-api"
Jul 06 18:18:17.036001 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.035750    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="52791fa0-69b5-4b46-b33b-80b0bf7e39ce" containerName="main"
Jul 06 18:18:17.036001 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.035762    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="687a17ca-0e05-4a71-8d22-309683e654ea" containerName="maas-api"
Jul 06 18:18:17.040145 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.040117    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:17.043651 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.043623    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-aa698f-serving-cert\""
Jul 06 18:18:17.052447 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.052414    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"]
Jul 06 18:18:17.205034 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.204986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3826cf9-11da-4bea-b9a3-a52c95ced88d-openshift-service-ca\") pod \"maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2\" (UID: \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\") " pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:17.205217 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.205067    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrs4g\" (UniqueName: \"kubernetes.io/projected/c3826cf9-11da-4bea-b9a3-a52c95ced88d-kube-api-access-mrs4g\") pod \"maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2\" (UID: \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\") " pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:17.205217 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.205093    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3826cf9-11da-4bea-b9a3-a52c95ced88d-maas-api-tls\") pod \"maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2\" (UID: \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\") " pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:17.306289 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.306181    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrs4g\" (UniqueName: \"kubernetes.io/projected/c3826cf9-11da-4bea-b9a3-a52c95ced88d-kube-api-access-mrs4g\") pod \"maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2\" (UID: \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\") " pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:17.306289 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.306236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3826cf9-11da-4bea-b9a3-a52c95ced88d-maas-api-tls\") pod \"maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2\" (UID: \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\") " pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:17.306573 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.306306    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3826cf9-11da-4bea-b9a3-a52c95ced88d-openshift-service-ca\") pod \"maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2\" (UID: \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\") " pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:17.306573 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:18:17.306421    2581 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-aa698f-serving-cert: secret "maas-api-e2e-api-a-aa698f-serving-cert" not found
Jul 06 18:18:17.306573 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:18:17.306484    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3826cf9-11da-4bea-b9a3-a52c95ced88d-maas-api-tls podName:c3826cf9-11da-4bea-b9a3-a52c95ced88d nodeName:}" failed. No retries permitted until 2026-07-06 18:18:17.80646784 +0000 UTC m=+2273.399922347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c3826cf9-11da-4bea-b9a3-a52c95ced88d-maas-api-tls") pod "maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2" (UID: "c3826cf9-11da-4bea-b9a3-a52c95ced88d") : secret "maas-api-e2e-api-a-aa698f-serving-cert" not found
Jul 06 18:18:17.307026 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.307008    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3826cf9-11da-4bea-b9a3-a52c95ced88d-openshift-service-ca\") pod \"maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2\" (UID: \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\") " pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:17.329681 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.329638    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrs4g\" (UniqueName: \"kubernetes.io/projected/c3826cf9-11da-4bea-b9a3-a52c95ced88d-kube-api-access-mrs4g\") pod \"maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2\" (UID: \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\") " pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:17.810550 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.810505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3826cf9-11da-4bea-b9a3-a52c95ced88d-maas-api-tls\") pod \"maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2\" (UID: \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\") " pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:17.813187 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.813159    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3826cf9-11da-4bea-b9a3-a52c95ced88d-maas-api-tls\") pod \"maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2\" (UID: \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\") " pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:17.953061 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:17.953020    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:18.106869 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:18.106841    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"]
Jul 06 18:18:18.109544 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:18:18.109513    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3826cf9_11da_4bea_b9a3_a52c95ced88d.slice/crio-ea188757bae611eaea9defd8ec8c2cf0ca7037ec5dbeeab418e6a5d463a0f2d6 WatchSource:0}: Error finding container ea188757bae611eaea9defd8ec8c2cf0ca7037ec5dbeeab418e6a5d463a0f2d6: Status 404 returned error can't find the container with id ea188757bae611eaea9defd8ec8c2cf0ca7037ec5dbeeab418e6a5d463a0f2d6
Jul 06 18:18:18.169318 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:18.169284    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2" event={"ID":"c3826cf9-11da-4bea-b9a3-a52c95ced88d","Type":"ContainerStarted","Data":"ea188757bae611eaea9defd8ec8c2cf0ca7037ec5dbeeab418e6a5d463a0f2d6"}
Jul 06 18:18:19.176036 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:19.175999    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2" event={"ID":"c3826cf9-11da-4bea-b9a3-a52c95ced88d","Type":"ContainerStarted","Data":"283e892836240459e8d69617362c975498b38435377dae41b16daa013e006779"}
Jul 06 18:18:19.176560 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:19.176117    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:19.202569 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:19.202507    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2" podStartSLOduration=1.6303174070000002 podStartE2EDuration="2.202490809s" totalImagesPullingTime="572.173402ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:19.2008108 +0000 UTC m=+2274.794265331" watchObservedRunningTime="2026-07-06 18:18:19.202490809 +0000 UTC m=+2274.795945339"
Jul 06 18:18:25.156117 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.156079    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"]
Jul 06 18:18:25.161354 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.161322    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:25.163777 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.163741    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-77c307-serving-cert\""
Jul 06 18:18:25.172741 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.172706    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"]
Jul 06 18:18:25.187575 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.187543    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:25.277448 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.277402    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fr87\" (UniqueName: \"kubernetes.io/projected/db3159ec-8ff6-4320-acd1-a3fc9d180a03-kube-api-access-5fr87\") pod \"maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd\" (UID: \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\") " pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:25.277448 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.277452    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3159ec-8ff6-4320-acd1-a3fc9d180a03-openshift-service-ca\") pod \"maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd\" (UID: \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\") " pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:25.277688 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.277584    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db3159ec-8ff6-4320-acd1-a3fc9d180a03-maas-api-tls\") pod \"maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd\" (UID: \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\") " pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:25.378218 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.378100    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3159ec-8ff6-4320-acd1-a3fc9d180a03-openshift-service-ca\") pod \"maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd\" (UID: \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\") " pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:25.378218 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.378181    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db3159ec-8ff6-4320-acd1-a3fc9d180a03-maas-api-tls\") pod \"maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd\" (UID: \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\") " pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:25.378491 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.378238    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fr87\" (UniqueName: \"kubernetes.io/projected/db3159ec-8ff6-4320-acd1-a3fc9d180a03-kube-api-access-5fr87\") pod \"maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd\" (UID: \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\") " pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:25.378491 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:18:25.378371    2581 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-77c307-serving-cert: secret "maas-api-e2e-api-b-77c307-serving-cert" not found
Jul 06 18:18:25.378491 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:18:25.378444    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db3159ec-8ff6-4320-acd1-a3fc9d180a03-maas-api-tls podName:db3159ec-8ff6-4320-acd1-a3fc9d180a03 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:25.878421709 +0000 UTC m=+2281.471876231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/db3159ec-8ff6-4320-acd1-a3fc9d180a03-maas-api-tls") pod "maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd" (UID: "db3159ec-8ff6-4320-acd1-a3fc9d180a03") : secret "maas-api-e2e-api-b-77c307-serving-cert" not found
Jul 06 18:18:25.378860 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.378837    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3159ec-8ff6-4320-acd1-a3fc9d180a03-openshift-service-ca\") pod \"maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd\" (UID: \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\") " pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:25.387877 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.387832    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fr87\" (UniqueName: \"kubernetes.io/projected/db3159ec-8ff6-4320-acd1-a3fc9d180a03-kube-api-access-5fr87\") pod \"maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd\" (UID: \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\") " pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:25.883396 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.883346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db3159ec-8ff6-4320-acd1-a3fc9d180a03-maas-api-tls\") pod \"maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd\" (UID: \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\") " pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:25.885883 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:25.885858    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db3159ec-8ff6-4320-acd1-a3fc9d180a03-maas-api-tls\") pod \"maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd\" (UID: \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\") " pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:26.076329 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:26.076287    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:26.239802 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:26.235085    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"]
Jul 06 18:18:27.217991 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:27.217941    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd" event={"ID":"db3159ec-8ff6-4320-acd1-a3fc9d180a03","Type":"ContainerStarted","Data":"90b71b86e1770a51c79c72297dc8bd4180f76c80bc356863573769a72fb575a0"}
Jul 06 18:18:27.218217 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:27.218017    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:27.218217 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:27.218037    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd" event={"ID":"db3159ec-8ff6-4320-acd1-a3fc9d180a03","Type":"ContainerStarted","Data":"4e7b63209787636ca0df14ce6e3a37ebb7a81d91a9fca8ca94104c43673b028b"}
Jul 06 18:18:27.235770 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:27.235714    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd" podStartSLOduration=1.877549969 podStartE2EDuration="2.235694706s" totalImagesPullingTime="358.144737ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:27.233946916 +0000 UTC m=+2282.827401451" watchObservedRunningTime="2026-07-06 18:18:27.235694706 +0000 UTC m=+2282.829149235"
Jul 06 18:18:33.229523 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:33.229487    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:38.141771 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:38.141715    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"]
Jul 06 18:18:38.142636 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:38.142599    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2" podUID="c3826cf9-11da-4bea-b9a3-a52c95ced88d" containerName="maas-api" containerID="cri-o://283e892836240459e8d69617362c975498b38435377dae41b16daa013e006779" gracePeriod=30
Jul 06 18:18:38.274594 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:38.274555    2581 generic.go:356] "Generic (PLEG): container finished" podID="c3826cf9-11da-4bea-b9a3-a52c95ced88d" containerID="283e892836240459e8d69617362c975498b38435377dae41b16daa013e006779" exitCode=0
Jul 06 18:18:38.274769 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:38.274628    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2" event={"ID":"c3826cf9-11da-4bea-b9a3-a52c95ced88d","Type":"ContainerDied","Data":"283e892836240459e8d69617362c975498b38435377dae41b16daa013e006779"}
Jul 06 18:18:38.402785 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:38.402711    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:38.495527 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:38.495470    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3826cf9-11da-4bea-b9a3-a52c95ced88d-maas-api-tls\") pod \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\" (UID: \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\") "
Jul 06 18:18:38.495723 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:38.495570    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrs4g\" (UniqueName: \"kubernetes.io/projected/c3826cf9-11da-4bea-b9a3-a52c95ced88d-kube-api-access-mrs4g\") pod \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\" (UID: \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\") "
Jul 06 18:18:38.495723 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:38.495632    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3826cf9-11da-4bea-b9a3-a52c95ced88d-openshift-service-ca\") pod \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\" (UID: \"c3826cf9-11da-4bea-b9a3-a52c95ced88d\") "
Jul 06 18:18:38.496033 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:38.496005    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3826cf9-11da-4bea-b9a3-a52c95ced88d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c3826cf9-11da-4bea-b9a3-a52c95ced88d" (UID: "c3826cf9-11da-4bea-b9a3-a52c95ced88d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:18:38.497779 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:38.497749    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3826cf9-11da-4bea-b9a3-a52c95ced88d-kube-api-access-mrs4g" (OuterVolumeSpecName: "kube-api-access-mrs4g") pod "c3826cf9-11da-4bea-b9a3-a52c95ced88d" (UID: "c3826cf9-11da-4bea-b9a3-a52c95ced88d"). InnerVolumeSpecName "kube-api-access-mrs4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:18:38.497852 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:38.497754    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3826cf9-11da-4bea-b9a3-a52c95ced88d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c3826cf9-11da-4bea-b9a3-a52c95ced88d" (UID: "c3826cf9-11da-4bea-b9a3-a52c95ced88d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:18:38.597123 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:38.597063    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrs4g\" (UniqueName: \"kubernetes.io/projected/c3826cf9-11da-4bea-b9a3-a52c95ced88d-kube-api-access-mrs4g\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:18:38.597123 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:38.597115    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3826cf9-11da-4bea-b9a3-a52c95ced88d-openshift-service-ca\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:18:38.597123 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:38.597130    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3826cf9-11da-4bea-b9a3-a52c95ced88d-maas-api-tls\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:18:39.280670 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:39.280629    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2" event={"ID":"c3826cf9-11da-4bea-b9a3-a52c95ced88d","Type":"ContainerDied","Data":"ea188757bae611eaea9defd8ec8c2cf0ca7037ec5dbeeab418e6a5d463a0f2d6"}
Jul 06 18:18:39.281144 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:39.280708    2581 scope.go:117] "RemoveContainer" containerID="283e892836240459e8d69617362c975498b38435377dae41b16daa013e006779"
Jul 06 18:18:39.281144 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:39.280643    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"
Jul 06 18:18:39.297203 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:39.297169    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"]
Jul 06 18:18:39.300106 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:39.300074    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-aa698f-56f575dc94-dfxf2"]
Jul 06 18:18:40.912870 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:40.912829    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3826cf9-11da-4bea-b9a3-a52c95ced88d" path="/var/lib/kubelet/pods/c3826cf9-11da-4bea-b9a3-a52c95ced88d/volumes"
Jul 06 18:18:44.861956 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:44.861918    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"]
Jul 06 18:18:44.862487 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:44.862184    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd" podUID="db3159ec-8ff6-4320-acd1-a3fc9d180a03" containerName="maas-api" containerID="cri-o://90b71b86e1770a51c79c72297dc8bd4180f76c80bc356863573769a72fb575a0" gracePeriod=30
Jul 06 18:18:45.119793 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.119710    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:45.155016 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.154982    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3159ec-8ff6-4320-acd1-a3fc9d180a03-openshift-service-ca\") pod \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\" (UID: \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\") "
Jul 06 18:18:45.155217 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.155066    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db3159ec-8ff6-4320-acd1-a3fc9d180a03-maas-api-tls\") pod \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\" (UID: \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\") "
Jul 06 18:18:45.155217 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.155091    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fr87\" (UniqueName: \"kubernetes.io/projected/db3159ec-8ff6-4320-acd1-a3fc9d180a03-kube-api-access-5fr87\") pod \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\" (UID: \"db3159ec-8ff6-4320-acd1-a3fc9d180a03\") "
Jul 06 18:18:45.155433 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.155412    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db3159ec-8ff6-4320-acd1-a3fc9d180a03-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "db3159ec-8ff6-4320-acd1-a3fc9d180a03" (UID: "db3159ec-8ff6-4320-acd1-a3fc9d180a03"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:18:45.157611 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.157575    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db3159ec-8ff6-4320-acd1-a3fc9d180a03-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "db3159ec-8ff6-4320-acd1-a3fc9d180a03" (UID: "db3159ec-8ff6-4320-acd1-a3fc9d180a03"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:18:45.157611 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.157599    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db3159ec-8ff6-4320-acd1-a3fc9d180a03-kube-api-access-5fr87" (OuterVolumeSpecName: "kube-api-access-5fr87") pod "db3159ec-8ff6-4320-acd1-a3fc9d180a03" (UID: "db3159ec-8ff6-4320-acd1-a3fc9d180a03"). InnerVolumeSpecName "kube-api-access-5fr87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:18:45.256536 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.256470    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db3159ec-8ff6-4320-acd1-a3fc9d180a03-openshift-service-ca\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:18:45.256536 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.256510    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db3159ec-8ff6-4320-acd1-a3fc9d180a03-maas-api-tls\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:18:45.256536 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.256524    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fr87\" (UniqueName: \"kubernetes.io/projected/db3159ec-8ff6-4320-acd1-a3fc9d180a03-kube-api-access-5fr87\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:18:45.315068 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.315026    2581 generic.go:356] "Generic (PLEG): container finished" podID="db3159ec-8ff6-4320-acd1-a3fc9d180a03" containerID="90b71b86e1770a51c79c72297dc8bd4180f76c80bc356863573769a72fb575a0" exitCode=0
Jul 06 18:18:45.315251 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.315076    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"
Jul 06 18:18:45.315251 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.315112    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd" event={"ID":"db3159ec-8ff6-4320-acd1-a3fc9d180a03","Type":"ContainerDied","Data":"90b71b86e1770a51c79c72297dc8bd4180f76c80bc356863573769a72fb575a0"}
Jul 06 18:18:45.315251 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.315173    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd" event={"ID":"db3159ec-8ff6-4320-acd1-a3fc9d180a03","Type":"ContainerDied","Data":"4e7b63209787636ca0df14ce6e3a37ebb7a81d91a9fca8ca94104c43673b028b"}
Jul 06 18:18:45.315251 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.315206    2581 scope.go:117] "RemoveContainer" containerID="90b71b86e1770a51c79c72297dc8bd4180f76c80bc356863573769a72fb575a0"
Jul 06 18:18:45.325823 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.325794    2581 scope.go:117] "RemoveContainer" containerID="90b71b86e1770a51c79c72297dc8bd4180f76c80bc356863573769a72fb575a0"
Jul 06 18:18:45.326395 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:18:45.326367    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90b71b86e1770a51c79c72297dc8bd4180f76c80bc356863573769a72fb575a0\": container with ID starting with 90b71b86e1770a51c79c72297dc8bd4180f76c80bc356863573769a72fb575a0 not found: ID does not exist" containerID="90b71b86e1770a51c79c72297dc8bd4180f76c80bc356863573769a72fb575a0"
Jul 06 18:18:45.326504 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.326405    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90b71b86e1770a51c79c72297dc8bd4180f76c80bc356863573769a72fb575a0"} err="failed to get container status \"90b71b86e1770a51c79c72297dc8bd4180f76c80bc356863573769a72fb575a0\": rpc error: code = NotFound desc = could not find container \"90b71b86e1770a51c79c72297dc8bd4180f76c80bc356863573769a72fb575a0\": container with ID starting with 90b71b86e1770a51c79c72297dc8bd4180f76c80bc356863573769a72fb575a0 not found: ID does not exist"
Jul 06 18:18:45.336806 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.336766    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"]
Jul 06 18:18:45.342994 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:45.342955    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-77c307-6cbbb8cc95-zf2nd"]
Jul 06 18:18:46.912489 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:46.912444    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db3159ec-8ff6-4320-acd1-a3fc9d180a03" path="/var/lib/kubelet/pods/db3159ec-8ff6-4320-acd1-a3fc9d180a03/volumes"
Jul 06 18:18:54.201515 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.201471    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"]
Jul 06 18:18:54.202101 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.202082    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3826cf9-11da-4bea-b9a3-a52c95ced88d" containerName="maas-api"
Jul 06 18:18:54.202155 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.202105    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3826cf9-11da-4bea-b9a3-a52c95ced88d" containerName="maas-api"
Jul 06 18:18:54.202155 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.202124    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db3159ec-8ff6-4320-acd1-a3fc9d180a03" containerName="maas-api"
Jul 06 18:18:54.202155 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.202132    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="db3159ec-8ff6-4320-acd1-a3fc9d180a03" containerName="maas-api"
Jul 06 18:18:54.202330 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.202233    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="db3159ec-8ff6-4320-acd1-a3fc9d180a03" containerName="maas-api"
Jul 06 18:18:54.202330 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.202246    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3826cf9-11da-4bea-b9a3-a52c95ced88d" containerName="maas-api"
Jul 06 18:18:54.207025 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.206994    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:18:54.209691 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.209668    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-91cb9f-serving-cert\""
Jul 06 18:18:54.215765 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.215733    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"]
Jul 06 18:18:54.333522 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.333479    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7l9z\" (UniqueName: \"kubernetes.io/projected/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-kube-api-access-g7l9z\") pod \"maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh\" (UID: \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\") " pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:18:54.333714 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.333538    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh\" (UID: \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\") " pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:18:54.333714 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.333638    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-maas-api-tls\") pod \"maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh\" (UID: \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\") " pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:18:54.435106 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.435060    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7l9z\" (UniqueName: \"kubernetes.io/projected/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-kube-api-access-g7l9z\") pod \"maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh\" (UID: \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\") " pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:18:54.435333 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.435119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh\" (UID: \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\") " pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:18:54.435333 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.435169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-maas-api-tls\") pod \"maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh\" (UID: \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\") " pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:18:54.435789 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.435763    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-openshift-service-ca\") pod \"maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh\" (UID: \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\") " pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:18:54.437719 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.437687    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-maas-api-tls\") pod \"maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh\" (UID: \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\") " pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:18:54.445493 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.445465    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7l9z\" (UniqueName: \"kubernetes.io/projected/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-kube-api-access-g7l9z\") pod \"maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh\" (UID: \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\") " pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:18:54.522781 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.522671    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:18:54.678243 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:54.678210    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"]
Jul 06 18:18:54.680369 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:18:54.680331    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ea67d5_aeb2_4fca_a67a_f652da9ce31c.slice/crio-53520152e2a1834714e0c53ee82134bd1a7039a62e365ac1f1b34a7a49d63775 WatchSource:0}: Error finding container 53520152e2a1834714e0c53ee82134bd1a7039a62e365ac1f1b34a7a49d63775: Status 404 returned error can't find the container with id 53520152e2a1834714e0c53ee82134bd1a7039a62e365ac1f1b34a7a49d63775
Jul 06 18:18:55.373978 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:55.373835    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh" event={"ID":"28ea67d5-aeb2-4fca-a67a-f652da9ce31c","Type":"ContainerStarted","Data":"c3a707e3703e0c77a968e5a40872837804989b743d68d809eb4d9ade935d580c"}
Jul 06 18:18:55.373978 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:55.373946    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh" event={"ID":"28ea67d5-aeb2-4fca-a67a-f652da9ce31c","Type":"ContainerStarted","Data":"53520152e2a1834714e0c53ee82134bd1a7039a62e365ac1f1b34a7a49d63775"}
Jul 06 18:18:55.373978 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:55.373978    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:18:55.394039 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:18:55.393981    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh" podStartSLOduration=0.973347041 podStartE2EDuration="1.393960429s" totalImagesPullingTime="420.613388ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:55.392666963 +0000 UTC m=+2310.986121514" watchObservedRunningTime="2026-07-06 18:18:55.393960429 +0000 UTC m=+2310.987414958"
Jul 06 18:19:01.384455 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:01.384420    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:19:02.237970 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:02.237927    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"]
Jul 06 18:19:02.241952 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:02.241920    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:19:02.244754 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:02.244725    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-2861c1-serving-cert\""
Jul 06 18:19:02.256782 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:02.256741    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"]
Jul 06 18:19:02.309531 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:02.309491    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh\" (UID: \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\") " pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:19:02.309723 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:02.309546    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdfk9\" (UniqueName: \"kubernetes.io/projected/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-kube-api-access-pdfk9\") pod \"maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh\" (UID: \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\") " pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:19:02.309723 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:02.309602    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-maas-api-tls\") pod \"maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh\" (UID: \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\") " pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:19:02.410213 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:02.410164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdfk9\" (UniqueName: \"kubernetes.io/projected/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-kube-api-access-pdfk9\") pod \"maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh\" (UID: \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\") " pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:19:02.410635 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:02.410234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-maas-api-tls\") pod \"maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh\" (UID: \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\") " pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:19:02.410635 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:02.410282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh\" (UID: \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\") " pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:19:02.410635 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:19:02.410386    2581 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-2861c1-serving-cert: secret "maas-api-e2e-shared-b-2861c1-serving-cert" not found
Jul 06 18:19:02.410635 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:19:02.410446    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-maas-api-tls podName:b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e nodeName:}" failed. No retries permitted until 2026-07-06 18:19:02.91043041 +0000 UTC m=+2318.503884922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-maas-api-tls") pod "maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh" (UID: "b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e") : secret "maas-api-e2e-shared-b-2861c1-serving-cert" not found
Jul 06 18:19:02.410890 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:02.410870    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh\" (UID: \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\") " pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:19:02.429655 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:02.429617    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdfk9\" (UniqueName: \"kubernetes.io/projected/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-kube-api-access-pdfk9\") pod \"maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh\" (UID: \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\") " pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:19:02.919704 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:02.919651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-maas-api-tls\") pod \"maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh\" (UID: \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\") " pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:19:02.922574 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:02.922535    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-maas-api-tls\") pod \"maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh\" (UID: \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\") " pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:19:03.157911 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:03.157860    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:19:03.308492 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:03.308460    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"]
Jul 06 18:19:03.310700 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:19:03.310666    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9d1b5e3_f61f_4f85_98f2_2f9f83953f1e.slice/crio-116e39a5432989340827836f82c5eed806564d9f55015421280f7287e31a3fd7 WatchSource:0}: Error finding container 116e39a5432989340827836f82c5eed806564d9f55015421280f7287e31a3fd7: Status 404 returned error can't find the container with id 116e39a5432989340827836f82c5eed806564d9f55015421280f7287e31a3fd7
Jul 06 18:19:03.416332 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:03.416291    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh" event={"ID":"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e","Type":"ContainerStarted","Data":"116e39a5432989340827836f82c5eed806564d9f55015421280f7287e31a3fd7"}
Jul 06 18:19:04.423096 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:04.423055    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh" event={"ID":"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e","Type":"ContainerStarted","Data":"7e8c96825e8f8a96a681eace55dd333a78e00ef28dd5dfdc833040a866a1b7ad"}
Jul 06 18:19:04.423520 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:04.423181    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:19:04.442248 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:04.442193    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh" podStartSLOduration=2.058759737 podStartE2EDuration="2.442172998s" totalImagesPullingTime="383.413261ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:19:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:04.438320169 +0000 UTC m=+2320.031774701" watchObservedRunningTime="2026-07-06 18:19:04.442172998 +0000 UTC m=+2320.035627528"
Jul 06 18:19:08.398703 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.398660    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"]
Jul 06 18:19:08.402530 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.402502    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.406294 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.406266    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-91cb9f\"/\"default-dockercfg-7nxvv\""
Jul 06 18:19:08.406478 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.406266    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-91cb9f\"/\"kube-root-ca.crt\""
Jul 06 18:19:08.406478 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.406308    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-91cb9f\"/\"openshift-service-ca.crt\""
Jul 06 18:19:08.406478 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.406330    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-91cb9f\"/\"auth-test-model-91cb9f-kserve-self-signed-certs\""
Jul 06 18:19:08.410416 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.410391    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"]
Jul 06 18:19:08.570931 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.570874    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrlf8\" (UniqueName: \"kubernetes.io/projected/12d15afb-6b63-4866-a28b-d583c8064a42-kube-api-access-hrlf8\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.571121 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.570986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-dshm\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.571121 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.571015    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-model-cache\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.571121 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.571038    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-tmp-dir\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.571121 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.571078    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-home\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.571121 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.571096    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12d15afb-6b63-4866-a28b-d583c8064a42-tls-certs\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.672364 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.672255    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-home\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.672364 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.672303    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12d15afb-6b63-4866-a28b-d583c8064a42-tls-certs\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.672364 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.672354    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrlf8\" (UniqueName: \"kubernetes.io/projected/12d15afb-6b63-4866-a28b-d583c8064a42-kube-api-access-hrlf8\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.672657 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.672412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-dshm\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.672657 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.672439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-model-cache\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.672657 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.672462    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-tmp-dir\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.672823 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.672713    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-home\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.672944 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.672923    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-tmp-dir\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.673006 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.672943    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-model-cache\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.674746 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.674718    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-dshm\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.674882 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.674863    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12d15afb-6b63-4866-a28b-d583c8064a42-tls-certs\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.680443 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.680404    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrlf8\" (UniqueName: \"kubernetes.io/projected/12d15afb-6b63-4866-a28b-d583c8064a42-kube-api-access-hrlf8\") pod \"auth-test-model-91cb9f-kserve-64868b4455-tvgdm\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") " pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.712564 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.712521    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:08.859259 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:08.859230    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"]
Jul 06 18:19:08.861586 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:19:08.861549    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12d15afb_6b63_4866_a28b_d583c8064a42.slice/crio-5d7677d208d6d1f56db3dd0236206d0c95ebfe26144511325bad35186b5b9910 WatchSource:0}: Error finding container 5d7677d208d6d1f56db3dd0236206d0c95ebfe26144511325bad35186b5b9910: Status 404 returned error can't find the container with id 5d7677d208d6d1f56db3dd0236206d0c95ebfe26144511325bad35186b5b9910
Jul 06 18:19:09.450804 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:09.450761    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm" event={"ID":"12d15afb-6b63-4866-a28b-d583c8064a42","Type":"ContainerStarted","Data":"85bba9a69d199824cf33848fd99b032458e38e87ca44f632266083b8a01321d4"}
Jul 06 18:19:09.450804 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:09.450811    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm" event={"ID":"12d15afb-6b63-4866-a28b-d583c8064a42","Type":"ContainerStarted","Data":"5d7677d208d6d1f56db3dd0236206d0c95ebfe26144511325bad35186b5b9910"}
Jul 06 18:19:09.468049 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:09.467987    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm" podStartSLOduration=1.467968669 podStartE2EDuration="1.467968669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:09.466211868 +0000 UTC m=+2325.059666397" watchObservedRunningTime="2026-07-06 18:19:09.467968669 +0000 UTC m=+2325.061423199"
Jul 06 18:19:10.432554 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:10.432518    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:19:18.713556 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:18.713453    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:18.713556 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:18.713515    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:18.726603 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:18.726567    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:19.513284 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:19.513248    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:19:36.795167 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:36.795126    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"]
Jul 06 18:19:36.799317 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:36.799290    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:36.803250 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:36.803213    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2861c1\"/\"auth-test-model-2861c1-kserve-self-signed-certs\""
Jul 06 18:19:36.803420 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:36.803271    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2861c1\"/\"kube-root-ca.crt\""
Jul 06 18:19:36.803420 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:36.803219    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2861c1\"/\"openshift-service-ca.crt\""
Jul 06 18:19:36.803420 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:36.803390    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2861c1\"/\"default-dockercfg-pvs5b\""
Jul 06 18:19:36.808059 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:36.808032    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"]
Jul 06 18:19:36.903819 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:36.903772    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-model-cache\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:36.904067 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:36.903845    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpj89\" (UniqueName: \"kubernetes.io/projected/d170ddcc-8033-4f95-b608-c15594306a58-kube-api-access-kpj89\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:36.904067 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:36.903884    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-dshm\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:36.904067 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:36.903985    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-tmp-dir\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:36.904067 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:36.904017    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d170ddcc-8033-4f95-b608-c15594306a58-tls-certs\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:36.904067 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:36.904039    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-home\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:37.005336 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.005295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-model-cache\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:37.005540 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.005358    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpj89\" (UniqueName: \"kubernetes.io/projected/d170ddcc-8033-4f95-b608-c15594306a58-kube-api-access-kpj89\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:37.005540 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.005381    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-dshm\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:37.005540 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.005404    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-tmp-dir\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:37.005540 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.005419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d170ddcc-8033-4f95-b608-c15594306a58-tls-certs\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:37.005540 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.005439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-home\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:37.005799 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.005770    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-model-cache\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:37.005865 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.005812    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-tmp-dir\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:37.005865 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.005851    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-home\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:37.007806 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.007782    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-dshm\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:37.008081 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.008063    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d170ddcc-8033-4f95-b608-c15594306a58-tls-certs\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:37.013763 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.013732    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpj89\" (UniqueName: \"kubernetes.io/projected/d170ddcc-8033-4f95-b608-c15594306a58-kube-api-access-kpj89\") pod \"auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") " pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:37.111382 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.111285    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:37.255797 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.255756    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"]
Jul 06 18:19:37.259209 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:19:37.259134    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd170ddcc_8033_4f95_b608_c15594306a58.slice/crio-9a00272a96b2b850c59865faf9516745497d29b74e9d114cc045e5fd07eb1371 WatchSource:0}: Error finding container 9a00272a96b2b850c59865faf9516745497d29b74e9d114cc045e5fd07eb1371: Status 404 returned error can't find the container with id 9a00272a96b2b850c59865faf9516745497d29b74e9d114cc045e5fd07eb1371
Jul 06 18:19:37.595522 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.595483    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb" event={"ID":"d170ddcc-8033-4f95-b608-c15594306a58","Type":"ContainerStarted","Data":"cdb4c58d2a2ed6402f3a62ac5209bebebf57b2dfc7e78f095c47314d7bdc1c40"}
Jul 06 18:19:37.595725 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.595533    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb" event={"ID":"d170ddcc-8033-4f95-b608-c15594306a58","Type":"ContainerStarted","Data":"9a00272a96b2b850c59865faf9516745497d29b74e9d114cc045e5fd07eb1371"}
Jul 06 18:19:37.612275 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:37.612214    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb" podStartSLOduration=1.61219644 podStartE2EDuration="1.61219644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:19:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:37.610695247 +0000 UTC m=+2353.204149778" watchObservedRunningTime="2026-07-06 18:19:37.61219644 +0000 UTC m=+2353.205650989"
Jul 06 18:19:47.111565 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:47.111512    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:47.111565 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:47.111580    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:47.124583 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:47.124553    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:19:47.656773 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:19:47.656741    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:21:32.681713 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:32.681662    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"]
Jul 06 18:21:33.697278 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:33.697246    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"]
Jul 06 18:21:47.702630 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:47.702589    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm" podUID="12d15afb-6b63-4866-a28b-d583c8064a42" containerName="main" containerID="cri-o://85bba9a69d199824cf33848fd99b032458e38e87ca44f632266083b8a01321d4" gracePeriod=45
Jul 06 18:21:47.962560 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:47.962485    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:21:48.091113 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.091070    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-home\") pod \"12d15afb-6b63-4866-a28b-d583c8064a42\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") "
Jul 06 18:21:48.091293 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.091129    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-dshm\") pod \"12d15afb-6b63-4866-a28b-d583c8064a42\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") "
Jul 06 18:21:48.091293 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.091149    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12d15afb-6b63-4866-a28b-d583c8064a42-tls-certs\") pod \"12d15afb-6b63-4866-a28b-d583c8064a42\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") "
Jul 06 18:21:48.091293 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.091169    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrlf8\" (UniqueName: \"kubernetes.io/projected/12d15afb-6b63-4866-a28b-d583c8064a42-kube-api-access-hrlf8\") pod \"12d15afb-6b63-4866-a28b-d583c8064a42\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") "
Jul 06 18:21:48.091293 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.091205    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-model-cache\") pod \"12d15afb-6b63-4866-a28b-d583c8064a42\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") "
Jul 06 18:21:48.091293 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.091220    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-tmp-dir\") pod \"12d15afb-6b63-4866-a28b-d583c8064a42\" (UID: \"12d15afb-6b63-4866-a28b-d583c8064a42\") "
Jul 06 18:21:48.091582 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.091382    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-home" (OuterVolumeSpecName: "home") pod "12d15afb-6b63-4866-a28b-d583c8064a42" (UID: "12d15afb-6b63-4866-a28b-d583c8064a42"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:21:48.091582 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.091484    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-model-cache" (OuterVolumeSpecName: "model-cache") pod "12d15afb-6b63-4866-a28b-d583c8064a42" (UID: "12d15afb-6b63-4866-a28b-d583c8064a42"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:21:48.091582 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.091556    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "12d15afb-6b63-4866-a28b-d583c8064a42" (UID: "12d15afb-6b63-4866-a28b-d583c8064a42"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:21:48.093481 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.093440    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-dshm" (OuterVolumeSpecName: "dshm") pod "12d15afb-6b63-4866-a28b-d583c8064a42" (UID: "12d15afb-6b63-4866-a28b-d583c8064a42"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:21:48.093481 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.093444    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12d15afb-6b63-4866-a28b-d583c8064a42-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "12d15afb-6b63-4866-a28b-d583c8064a42" (UID: "12d15afb-6b63-4866-a28b-d583c8064a42"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:21:48.093655 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.093501    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12d15afb-6b63-4866-a28b-d583c8064a42-kube-api-access-hrlf8" (OuterVolumeSpecName: "kube-api-access-hrlf8") pod "12d15afb-6b63-4866-a28b-d583c8064a42" (UID: "12d15afb-6b63-4866-a28b-d583c8064a42"). InnerVolumeSpecName "kube-api-access-hrlf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:21:48.192685 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.192639    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-home\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:21:48.192685 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.192673    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-dshm\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:21:48.192685 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.192682    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12d15afb-6b63-4866-a28b-d583c8064a42-tls-certs\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:21:48.192685 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.192692    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrlf8\" (UniqueName: \"kubernetes.io/projected/12d15afb-6b63-4866-a28b-d583c8064a42-kube-api-access-hrlf8\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:21:48.192992 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.192701    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-model-cache\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:21:48.192992 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.192709    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12d15afb-6b63-4866-a28b-d583c8064a42-tmp-dir\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:21:48.278386 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.278296    2581 generic.go:356] "Generic (PLEG): container finished" podID="12d15afb-6b63-4866-a28b-d583c8064a42" containerID="85bba9a69d199824cf33848fd99b032458e38e87ca44f632266083b8a01321d4" exitCode=0
Jul 06 18:21:48.278386 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.278339    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"
Jul 06 18:21:48.278386 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.278359    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm" event={"ID":"12d15afb-6b63-4866-a28b-d583c8064a42","Type":"ContainerDied","Data":"85bba9a69d199824cf33848fd99b032458e38e87ca44f632266083b8a01321d4"}
Jul 06 18:21:48.278386 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.278399    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm" event={"ID":"12d15afb-6b63-4866-a28b-d583c8064a42","Type":"ContainerDied","Data":"5d7677d208d6d1f56db3dd0236206d0c95ebfe26144511325bad35186b5b9910"}
Jul 06 18:21:48.278728 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.278416    2581 scope.go:117] "RemoveContainer" containerID="85bba9a69d199824cf33848fd99b032458e38e87ca44f632266083b8a01321d4"
Jul 06 18:21:48.290187 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.290161    2581 scope.go:117] "RemoveContainer" containerID="85bba9a69d199824cf33848fd99b032458e38e87ca44f632266083b8a01321d4"
Jul 06 18:21:48.290565 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:21:48.290545    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85bba9a69d199824cf33848fd99b032458e38e87ca44f632266083b8a01321d4\": container with ID starting with 85bba9a69d199824cf33848fd99b032458e38e87ca44f632266083b8a01321d4 not found: ID does not exist" containerID="85bba9a69d199824cf33848fd99b032458e38e87ca44f632266083b8a01321d4"
Jul 06 18:21:48.290637 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.290576    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85bba9a69d199824cf33848fd99b032458e38e87ca44f632266083b8a01321d4"} err="failed to get container status \"85bba9a69d199824cf33848fd99b032458e38e87ca44f632266083b8a01321d4\": rpc error: code = NotFound desc = could not find container \"85bba9a69d199824cf33848fd99b032458e38e87ca44f632266083b8a01321d4\": container with ID starting with 85bba9a69d199824cf33848fd99b032458e38e87ca44f632266083b8a01321d4 not found: ID does not exist"
Jul 06 18:21:48.303933 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.303884    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"]
Jul 06 18:21:48.306829 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.306800    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/auth-test-model-91cb9f-kserve-64868b4455-tvgdm"]
Jul 06 18:21:48.718151 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.718110    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb" podUID="d170ddcc-8033-4f95-b608-c15594306a58" containerName="main" containerID="cri-o://cdb4c58d2a2ed6402f3a62ac5209bebebf57b2dfc7e78f095c47314d7bdc1c40" gracePeriod=45
Jul 06 18:21:48.918124 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.918089    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12d15afb-6b63-4866-a28b-d583c8064a42" path="/var/lib/kubelet/pods/12d15afb-6b63-4866-a28b-d583c8064a42/volumes"
Jul 06 18:21:48.980821 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:48.980747    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:21:49.101627 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.101590    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-dshm\") pod \"d170ddcc-8033-4f95-b608-c15594306a58\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") "
Jul 06 18:21:49.101821 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.101651    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-tmp-dir\") pod \"d170ddcc-8033-4f95-b608-c15594306a58\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") "
Jul 06 18:21:49.101821 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.101714    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-model-cache\") pod \"d170ddcc-8033-4f95-b608-c15594306a58\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") "
Jul 06 18:21:49.101821 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.101763    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-home\") pod \"d170ddcc-8033-4f95-b608-c15594306a58\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") "
Jul 06 18:21:49.101821 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.101788    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpj89\" (UniqueName: \"kubernetes.io/projected/d170ddcc-8033-4f95-b608-c15594306a58-kube-api-access-kpj89\") pod \"d170ddcc-8033-4f95-b608-c15594306a58\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") "
Jul 06 18:21:49.101821 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.101803    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d170ddcc-8033-4f95-b608-c15594306a58-tls-certs\") pod \"d170ddcc-8033-4f95-b608-c15594306a58\" (UID: \"d170ddcc-8033-4f95-b608-c15594306a58\") "
Jul 06 18:21:49.102111 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.101991    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d170ddcc-8033-4f95-b608-c15594306a58" (UID: "d170ddcc-8033-4f95-b608-c15594306a58"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:21:49.102111 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.102074    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-home" (OuterVolumeSpecName: "home") pod "d170ddcc-8033-4f95-b608-c15594306a58" (UID: "d170ddcc-8033-4f95-b608-c15594306a58"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:21:49.102180 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.102118    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-model-cache" (OuterVolumeSpecName: "model-cache") pod "d170ddcc-8033-4f95-b608-c15594306a58" (UID: "d170ddcc-8033-4f95-b608-c15594306a58"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:21:49.104093 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.104061    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d170ddcc-8033-4f95-b608-c15594306a58-kube-api-access-kpj89" (OuterVolumeSpecName: "kube-api-access-kpj89") pod "d170ddcc-8033-4f95-b608-c15594306a58" (UID: "d170ddcc-8033-4f95-b608-c15594306a58"). InnerVolumeSpecName "kube-api-access-kpj89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:21:49.104209 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.104106    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-dshm" (OuterVolumeSpecName: "dshm") pod "d170ddcc-8033-4f95-b608-c15594306a58" (UID: "d170ddcc-8033-4f95-b608-c15594306a58"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:21:49.104209 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.104134    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d170ddcc-8033-4f95-b608-c15594306a58-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d170ddcc-8033-4f95-b608-c15594306a58" (UID: "d170ddcc-8033-4f95-b608-c15594306a58"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:21:49.202838 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.202796    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-model-cache\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:21:49.202838 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.202828    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-home\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:21:49.202838 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.202837    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpj89\" (UniqueName: \"kubernetes.io/projected/d170ddcc-8033-4f95-b608-c15594306a58-kube-api-access-kpj89\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:21:49.202838 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.202846    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d170ddcc-8033-4f95-b608-c15594306a58-tls-certs\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:21:49.202838 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.202854    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-dshm\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:21:49.203182 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.202863    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d170ddcc-8033-4f95-b608-c15594306a58-tmp-dir\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:21:49.284963 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.284848    2581 generic.go:356] "Generic (PLEG): container finished" podID="d170ddcc-8033-4f95-b608-c15594306a58" containerID="cdb4c58d2a2ed6402f3a62ac5209bebebf57b2dfc7e78f095c47314d7bdc1c40" exitCode=0
Jul 06 18:21:49.284963 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.284890    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"
Jul 06 18:21:49.284963 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.284938    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb" event={"ID":"d170ddcc-8033-4f95-b608-c15594306a58","Type":"ContainerDied","Data":"cdb4c58d2a2ed6402f3a62ac5209bebebf57b2dfc7e78f095c47314d7bdc1c40"}
Jul 06 18:21:49.285204 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.284982    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb" event={"ID":"d170ddcc-8033-4f95-b608-c15594306a58","Type":"ContainerDied","Data":"9a00272a96b2b850c59865faf9516745497d29b74e9d114cc045e5fd07eb1371"}
Jul 06 18:21:49.285204 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.285004    2581 scope.go:117] "RemoveContainer" containerID="cdb4c58d2a2ed6402f3a62ac5209bebebf57b2dfc7e78f095c47314d7bdc1c40"
Jul 06 18:21:49.294132 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.294110    2581 scope.go:117] "RemoveContainer" containerID="cdb4c58d2a2ed6402f3a62ac5209bebebf57b2dfc7e78f095c47314d7bdc1c40"
Jul 06 18:21:49.294465 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:21:49.294446    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdb4c58d2a2ed6402f3a62ac5209bebebf57b2dfc7e78f095c47314d7bdc1c40\": container with ID starting with cdb4c58d2a2ed6402f3a62ac5209bebebf57b2dfc7e78f095c47314d7bdc1c40 not found: ID does not exist" containerID="cdb4c58d2a2ed6402f3a62ac5209bebebf57b2dfc7e78f095c47314d7bdc1c40"
Jul 06 18:21:49.294516 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.294474    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdb4c58d2a2ed6402f3a62ac5209bebebf57b2dfc7e78f095c47314d7bdc1c40"} err="failed to get container status \"cdb4c58d2a2ed6402f3a62ac5209bebebf57b2dfc7e78f095c47314d7bdc1c40\": rpc error: code = NotFound desc = could not find container \"cdb4c58d2a2ed6402f3a62ac5209bebebf57b2dfc7e78f095c47314d7bdc1c40\": container with ID starting with cdb4c58d2a2ed6402f3a62ac5209bebebf57b2dfc7e78f095c47314d7bdc1c40 not found: ID does not exist"
Jul 06 18:21:49.305714 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.305680    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"]
Jul 06 18:21:49.309337 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:49.309303    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/auth-test-model-2861c1-kserve-649bbf6fcf-9mpfb"]
Jul 06 18:21:50.097260 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.097223    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"]
Jul 06 18:21:50.097767 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.097701    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12d15afb-6b63-4866-a28b-d583c8064a42" containerName="main"
Jul 06 18:21:50.097767 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.097720    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="12d15afb-6b63-4866-a28b-d583c8064a42" containerName="main"
Jul 06 18:21:50.097767 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.097748    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d170ddcc-8033-4f95-b608-c15594306a58" containerName="main"
Jul 06 18:21:50.097767 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.097753    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d170ddcc-8033-4f95-b608-c15594306a58" containerName="main"
Jul 06 18:21:50.097996 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.097832    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d170ddcc-8033-4f95-b608-c15594306a58" containerName="main"
Jul 06 18:21:50.097996 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.097843    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="12d15afb-6b63-4866-a28b-d583c8064a42" containerName="main"
Jul 06 18:21:50.102503 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.102482    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.106594 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.106565    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-91cb9f\"/\"openshift-service-ca.crt\""
Jul 06 18:21:50.106758 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.106725    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-91cb9f\"/\"kube-root-ca.crt\""
Jul 06 18:21:50.106758 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.106739    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-91cb9f\"/\"sub-test-model-91cb9f-kserve-self-signed-certs\""
Jul 06 18:21:50.106956 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.106937    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-91cb9f\"/\"default-dockercfg-7nxvv\""
Jul 06 18:21:50.110100 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.110073    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"]
Jul 06 18:21:50.210960 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.210920    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-tmp-dir\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.211152 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.211000    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-tls-certs\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.211152 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.211033    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-model-cache\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.211152 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.211061    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-dshm\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.211152 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.211087    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqbdk\" (UniqueName: \"kubernetes.io/projected/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-kube-api-access-hqbdk\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.211152 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.211128    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-home\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.312437 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.312391    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-home\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.312663 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.312451    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-tmp-dir\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.312663 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.312523    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-tls-certs\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.312663 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.312555    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-model-cache\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.312663 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.312590    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-dshm\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.312663 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.312612    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqbdk\" (UniqueName: \"kubernetes.io/projected/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-kube-api-access-hqbdk\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.312957 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.312848    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-home\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.312957 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.312923    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-tmp-dir\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.313097 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.312967    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-model-cache\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.314968 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.314945    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-dshm\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.315134 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.315114    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-tls-certs\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.320672 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.320625    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqbdk\" (UniqueName: \"kubernetes.io/projected/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-kube-api-access-hqbdk\") pod \"sub-test-model-91cb9f-kserve-7bcc798b99-rwc48\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") " pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.415264 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.415157    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:21:50.561648 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.561619    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"]
Jul 06 18:21:50.563800 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:21:50.563775    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf5a6caa_8d0f_42a2_ba4b_b7e42bee82b8.slice/crio-90217c46ba5424b365067b8da20ee5228f5f68255007a5dce6843822ef19b513 WatchSource:0}: Error finding container 90217c46ba5424b365067b8da20ee5228f5f68255007a5dce6843822ef19b513: Status 404 returned error can't find the container with id 90217c46ba5424b365067b8da20ee5228f5f68255007a5dce6843822ef19b513
Jul 06 18:21:50.912910 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:50.912861    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d170ddcc-8033-4f95-b608-c15594306a58" path="/var/lib/kubelet/pods/d170ddcc-8033-4f95-b608-c15594306a58/volumes"
Jul 06 18:21:51.298406 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:51.298358    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48" event={"ID":"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8","Type":"ContainerStarted","Data":"3a9e08c0725948a6d0b8d38726b2aeb0bcddee78cc4d31db7d3793a8e4260439"}
Jul 06 18:21:51.298406 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:51.298414    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48" event={"ID":"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8","Type":"ContainerStarted","Data":"90217c46ba5424b365067b8da20ee5228f5f68255007a5dce6843822ef19b513"}
Jul 06 18:21:51.315350 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:21:51.315298    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48" podStartSLOduration=1.315280985 podStartE2EDuration="1.315280985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:21:51.314128836 +0000 UTC m=+2486.907583366" watchObservedRunningTime="2026-07-06 18:21:51.315280985 +0000 UTC m=+2486.908735515"
Jul 06 18:22:00.415846 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:00.415801    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:22:00.416504 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:00.415864    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:22:00.428527 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:00.428493    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:22:01.360810 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:01.360778    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:22:10.798246 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.798209    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"]
Jul 06 18:22:10.801852 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.801823    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.805870 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.805840    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2861c1\"/\"sub-test-model-2861c1-kserve-self-signed-certs\""
Jul 06 18:22:10.805870 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.805861    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2861c1\"/\"default-dockercfg-pvs5b\""
Jul 06 18:22:10.806118 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.805844    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2861c1\"/\"kube-root-ca.crt\""
Jul 06 18:22:10.806118 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.805924    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2861c1\"/\"openshift-service-ca.crt\""
Jul 06 18:22:10.812859 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.812828    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"]
Jul 06 18:22:10.896220 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.896182    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-dshm\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.896220 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.896221    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxj9r\" (UniqueName: \"kubernetes.io/projected/b0adbcea-c195-4b85-abb1-fdc7e670827e-kube-api-access-nxj9r\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.896450 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.896303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-tmp-dir\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.896450 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.896353    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-home\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.896450 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.896377    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0adbcea-c195-4b85-abb1-fdc7e670827e-tls-certs\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.896450 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.896443    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-model-cache\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.997253 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.997214    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-tmp-dir\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.997459 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.997273    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-home\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.997459 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.997295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0adbcea-c195-4b85-abb1-fdc7e670827e-tls-certs\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.997459 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.997339    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-model-cache\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.997624 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.997482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-dshm\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.997624 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.997528    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxj9r\" (UniqueName: \"kubernetes.io/projected/b0adbcea-c195-4b85-abb1-fdc7e670827e-kube-api-access-nxj9r\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.997741 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.997649    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-tmp-dir\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.997741 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.997682    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-model-cache\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.997835 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.997761    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-home\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.999773 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.999752    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-dshm\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:10.999977 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:10.999959    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0adbcea-c195-4b85-abb1-fdc7e670827e-tls-certs\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:11.005244 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:11.005211    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxj9r\" (UniqueName: \"kubernetes.io/projected/b0adbcea-c195-4b85-abb1-fdc7e670827e-kube-api-access-nxj9r\") pod \"sub-test-model-2861c1-kserve-8585575b5-76qlq\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") " pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:11.114168 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:11.114072    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:11.258210 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:11.258179    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"]
Jul 06 18:22:11.260721 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:22:11.260686    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0adbcea_c195_4b85_abb1_fdc7e670827e.slice/crio-d2b31ac2471c646b056509a9a8e0f5e5183f556136fa84448ddef46b2727a53b WatchSource:0}: Error finding container d2b31ac2471c646b056509a9a8e0f5e5183f556136fa84448ddef46b2727a53b: Status 404 returned error can't find the container with id d2b31ac2471c646b056509a9a8e0f5e5183f556136fa84448ddef46b2727a53b
Jul 06 18:22:11.404051 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:11.403942    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq" event={"ID":"b0adbcea-c195-4b85-abb1-fdc7e670827e","Type":"ContainerStarted","Data":"79139fcecfcd3d7cdf7296cf0fe28c865aeff2d3cc8994950ceaec25556fe4f2"}
Jul 06 18:22:11.404051 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:11.403994    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq" event={"ID":"b0adbcea-c195-4b85-abb1-fdc7e670827e","Type":"ContainerStarted","Data":"d2b31ac2471c646b056509a9a8e0f5e5183f556136fa84448ddef46b2727a53b"}
Jul 06 18:22:11.419679 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:11.419611    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq" podStartSLOduration=1.419593359 podStartE2EDuration="1.419593359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:22:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:22:11.418684415 +0000 UTC m=+2507.012138961" watchObservedRunningTime="2026-07-06 18:22:11.419593359 +0000 UTC m=+2507.013047889"
Jul 06 18:22:21.114763 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:21.114663    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:21.114763 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:21.114714    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:21.127252 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:21.127220    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:21.467411 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:21.467383    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:22:47.171853 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:47.171811    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"]
Jul 06 18:22:48.096065 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:22:48.096021    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"]
Jul 06 18:23:02.192826 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.192786    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48" podUID="bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8" containerName="main" containerID="cri-o://3a9e08c0725948a6d0b8d38726b2aeb0bcddee78cc4d31db7d3793a8e4260439" gracePeriod=45
Jul 06 18:23:02.467089 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.467056    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:23:02.657501 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.657456    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqbdk\" (UniqueName: \"kubernetes.io/projected/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-kube-api-access-hqbdk\") pod \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") "
Jul 06 18:23:02.657501 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.657504    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-tls-certs\") pod \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") "
Jul 06 18:23:02.657770 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.657571    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-dshm\") pod \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") "
Jul 06 18:23:02.657770 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.657603    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-tmp-dir\") pod \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") "
Jul 06 18:23:02.657770 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.657629    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-model-cache\") pod \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") "
Jul 06 18:23:02.657770 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.657659    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-home\") pod \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\" (UID: \"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8\") "
Jul 06 18:23:02.658026 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.657970    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8" (UID: "bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:23:02.658085 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.658011    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-home" (OuterVolumeSpecName: "home") pod "bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8" (UID: "bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:23:02.658085 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.658058    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-model-cache" (OuterVolumeSpecName: "model-cache") pod "bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8" (UID: "bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:23:02.659757 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.659727    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8" (UID: "bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:23:02.659885 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.659755    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-dshm" (OuterVolumeSpecName: "dshm") pod "bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8" (UID: "bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:23:02.659962 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.659879    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-kube-api-access-hqbdk" (OuterVolumeSpecName: "kube-api-access-hqbdk") pod "bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8" (UID: "bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8"). InnerVolumeSpecName "kube-api-access-hqbdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:23:02.665303 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.665268    2581 generic.go:356] "Generic (PLEG): container finished" podID="bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8" containerID="3a9e08c0725948a6d0b8d38726b2aeb0bcddee78cc4d31db7d3793a8e4260439" exitCode=0
Jul 06 18:23:02.665473 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.665318    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48" event={"ID":"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8","Type":"ContainerDied","Data":"3a9e08c0725948a6d0b8d38726b2aeb0bcddee78cc4d31db7d3793a8e4260439"}
Jul 06 18:23:02.665473 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.665329    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"
Jul 06 18:23:02.665473 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.665358    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48" event={"ID":"bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8","Type":"ContainerDied","Data":"90217c46ba5424b365067b8da20ee5228f5f68255007a5dce6843822ef19b513"}
Jul 06 18:23:02.665473 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.665380    2581 scope.go:117] "RemoveContainer" containerID="3a9e08c0725948a6d0b8d38726b2aeb0bcddee78cc4d31db7d3793a8e4260439"
Jul 06 18:23:02.680259 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.680238    2581 scope.go:117] "RemoveContainer" containerID="3a9e08c0725948a6d0b8d38726b2aeb0bcddee78cc4d31db7d3793a8e4260439"
Jul 06 18:23:02.680592 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:23:02.680566    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a9e08c0725948a6d0b8d38726b2aeb0bcddee78cc4d31db7d3793a8e4260439\": container with ID starting with 3a9e08c0725948a6d0b8d38726b2aeb0bcddee78cc4d31db7d3793a8e4260439 not found: ID does not exist" containerID="3a9e08c0725948a6d0b8d38726b2aeb0bcddee78cc4d31db7d3793a8e4260439"
Jul 06 18:23:02.680671 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.680607    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a9e08c0725948a6d0b8d38726b2aeb0bcddee78cc4d31db7d3793a8e4260439"} err="failed to get container status \"3a9e08c0725948a6d0b8d38726b2aeb0bcddee78cc4d31db7d3793a8e4260439\": rpc error: code = NotFound desc = could not find container \"3a9e08c0725948a6d0b8d38726b2aeb0bcddee78cc4d31db7d3793a8e4260439\": container with ID starting with 3a9e08c0725948a6d0b8d38726b2aeb0bcddee78cc4d31db7d3793a8e4260439 not found: ID does not exist"
Jul 06 18:23:02.690973 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.690939    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"]
Jul 06 18:23:02.694212 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.694177    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/sub-test-model-91cb9f-kserve-7bcc798b99-rwc48"]
Jul 06 18:23:02.759013 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.758920    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqbdk\" (UniqueName: \"kubernetes.io/projected/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-kube-api-access-hqbdk\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:23:02.759013 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.758952    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-tls-certs\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:23:02.759013 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.758963    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-dshm\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:23:02.759013 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.758972    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-tmp-dir\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:23:02.759013 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.758979    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-model-cache\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:23:02.759013 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.758986    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8-home\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:23:02.912946 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:02.912910    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8" path="/var/lib/kubelet/pods/bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8/volumes"
Jul 06 18:23:03.117172 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.117042    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq" podUID="b0adbcea-c195-4b85-abb1-fdc7e670827e" containerName="main" containerID="cri-o://79139fcecfcd3d7cdf7296cf0fe28c865aeff2d3cc8994950ceaec25556fe4f2" gracePeriod=45
Jul 06 18:23:03.383773 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.383740    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:23:03.569410 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.569370    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0adbcea-c195-4b85-abb1-fdc7e670827e-tls-certs\") pod \"b0adbcea-c195-4b85-abb1-fdc7e670827e\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") "
Jul 06 18:23:03.569612 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.569487    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxj9r\" (UniqueName: \"kubernetes.io/projected/b0adbcea-c195-4b85-abb1-fdc7e670827e-kube-api-access-nxj9r\") pod \"b0adbcea-c195-4b85-abb1-fdc7e670827e\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") "
Jul 06 18:23:03.569612 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.569529    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-dshm\") pod \"b0adbcea-c195-4b85-abb1-fdc7e670827e\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") "
Jul 06 18:23:03.569612 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.569561    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-model-cache\") pod \"b0adbcea-c195-4b85-abb1-fdc7e670827e\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") "
Jul 06 18:23:03.569612 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.569599    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-home\") pod \"b0adbcea-c195-4b85-abb1-fdc7e670827e\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") "
Jul 06 18:23:03.570182 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.569635    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-tmp-dir\") pod \"b0adbcea-c195-4b85-abb1-fdc7e670827e\" (UID: \"b0adbcea-c195-4b85-abb1-fdc7e670827e\") "
Jul 06 18:23:03.570182 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.569791    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-model-cache" (OuterVolumeSpecName: "model-cache") pod "b0adbcea-c195-4b85-abb1-fdc7e670827e" (UID: "b0adbcea-c195-4b85-abb1-fdc7e670827e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:23:03.570182 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.569864    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-home" (OuterVolumeSpecName: "home") pod "b0adbcea-c195-4b85-abb1-fdc7e670827e" (UID: "b0adbcea-c195-4b85-abb1-fdc7e670827e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:23:03.570182 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.569982    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b0adbcea-c195-4b85-abb1-fdc7e670827e" (UID: "b0adbcea-c195-4b85-abb1-fdc7e670827e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:23:03.570427 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.570360    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-model-cache\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:23:03.570427 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.570379    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-home\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:23:03.570427 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.570393    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-tmp-dir\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:23:03.571935 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.571879    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0adbcea-c195-4b85-abb1-fdc7e670827e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b0adbcea-c195-4b85-abb1-fdc7e670827e" (UID: "b0adbcea-c195-4b85-abb1-fdc7e670827e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:23:03.572230 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.572203    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-dshm" (OuterVolumeSpecName: "dshm") pod "b0adbcea-c195-4b85-abb1-fdc7e670827e" (UID: "b0adbcea-c195-4b85-abb1-fdc7e670827e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:23:03.572322 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.572249    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0adbcea-c195-4b85-abb1-fdc7e670827e-kube-api-access-nxj9r" (OuterVolumeSpecName: "kube-api-access-nxj9r") pod "b0adbcea-c195-4b85-abb1-fdc7e670827e" (UID: "b0adbcea-c195-4b85-abb1-fdc7e670827e"). InnerVolumeSpecName "kube-api-access-nxj9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:23:03.670862 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.670762    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxj9r\" (UniqueName: \"kubernetes.io/projected/b0adbcea-c195-4b85-abb1-fdc7e670827e-kube-api-access-nxj9r\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:23:03.670862 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.670789    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0adbcea-c195-4b85-abb1-fdc7e670827e-dshm\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:23:03.670862 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.670798    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0adbcea-c195-4b85-abb1-fdc7e670827e-tls-certs\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:23:03.672374 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.672346    2581 generic.go:356] "Generic (PLEG): container finished" podID="b0adbcea-c195-4b85-abb1-fdc7e670827e" containerID="79139fcecfcd3d7cdf7296cf0fe28c865aeff2d3cc8994950ceaec25556fe4f2" exitCode=0
Jul 06 18:23:03.672510 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.672394    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"
Jul 06 18:23:03.672510 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.672405    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq" event={"ID":"b0adbcea-c195-4b85-abb1-fdc7e670827e","Type":"ContainerDied","Data":"79139fcecfcd3d7cdf7296cf0fe28c865aeff2d3cc8994950ceaec25556fe4f2"}
Jul 06 18:23:03.672510 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.672436    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq" event={"ID":"b0adbcea-c195-4b85-abb1-fdc7e670827e","Type":"ContainerDied","Data":"d2b31ac2471c646b056509a9a8e0f5e5183f556136fa84448ddef46b2727a53b"}
Jul 06 18:23:03.672510 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.672452    2581 scope.go:117] "RemoveContainer" containerID="79139fcecfcd3d7cdf7296cf0fe28c865aeff2d3cc8994950ceaec25556fe4f2"
Jul 06 18:23:03.682064 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.682043    2581 scope.go:117] "RemoveContainer" containerID="79139fcecfcd3d7cdf7296cf0fe28c865aeff2d3cc8994950ceaec25556fe4f2"
Jul 06 18:23:03.682337 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:23:03.682318    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79139fcecfcd3d7cdf7296cf0fe28c865aeff2d3cc8994950ceaec25556fe4f2\": container with ID starting with 79139fcecfcd3d7cdf7296cf0fe28c865aeff2d3cc8994950ceaec25556fe4f2 not found: ID does not exist" containerID="79139fcecfcd3d7cdf7296cf0fe28c865aeff2d3cc8994950ceaec25556fe4f2"
Jul 06 18:23:03.682393 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.682345    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79139fcecfcd3d7cdf7296cf0fe28c865aeff2d3cc8994950ceaec25556fe4f2"} err="failed to get container status \"79139fcecfcd3d7cdf7296cf0fe28c865aeff2d3cc8994950ceaec25556fe4f2\": rpc error: code = NotFound desc = could not find container \"79139fcecfcd3d7cdf7296cf0fe28c865aeff2d3cc8994950ceaec25556fe4f2\": container with ID starting with 79139fcecfcd3d7cdf7296cf0fe28c865aeff2d3cc8994950ceaec25556fe4f2 not found: ID does not exist"
Jul 06 18:23:03.706376 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.706331    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"]
Jul 06 18:23:03.708023 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:03.707994    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/sub-test-model-2861c1-kserve-8585575b5-76qlq"]
Jul 06 18:23:04.913142 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:04.913105    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0adbcea-c195-4b85-abb1-fdc7e670827e" path="/var/lib/kubelet/pods/b0adbcea-c195-4b85-abb1-fdc7e670827e/volumes"
Jul 06 18:23:09.304236 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.304198    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"]
Jul 06 18:23:09.304759 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.304717    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0adbcea-c195-4b85-abb1-fdc7e670827e" containerName="main"
Jul 06 18:23:09.304759 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.304738    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0adbcea-c195-4b85-abb1-fdc7e670827e" containerName="main"
Jul 06 18:23:09.304858 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.304775    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8" containerName="main"
Jul 06 18:23:09.304858 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.304784    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8" containerName="main"
Jul 06 18:23:09.304984 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.304876    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0adbcea-c195-4b85-abb1-fdc7e670827e" containerName="main"
Jul 06 18:23:09.304984 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.304908    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf5a6caa-8d0f-42a2-ba4b-b7e42bee82b8" containerName="main"
Jul 06 18:23:09.307880 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.307860    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.310667 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.310643    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-91cb9f\"/\"kube-root-ca.crt\""
Jul 06 18:23:09.311978 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.311954    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-91cb9f\"/\"rate-test-model-91cb9f-kserve-self-signed-certs\""
Jul 06 18:23:09.312077 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.311997    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-91cb9f\"/\"default-dockercfg-7nxvv\""
Jul 06 18:23:09.312077 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.312018    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-91cb9f\"/\"openshift-service-ca.crt\""
Jul 06 18:23:09.312311 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.312294    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c228df05-af0d-4143-8fe9-80f7bae2f943-tls-certs\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.312400 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.312320    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ffm4\" (UniqueName: \"kubernetes.io/projected/c228df05-af0d-4143-8fe9-80f7bae2f943-kube-api-access-6ffm4\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.312400 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.312353    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-tmp-dir\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.312508 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.312483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-home\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.312566 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.312522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-dshm\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.312628 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.312606    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-model-cache\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.317793 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.317762    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"]
Jul 06 18:23:09.413373 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.413335    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-home\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.413373 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.413374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-dshm\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.413616 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.413431    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-model-cache\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.413616 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.413469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c228df05-af0d-4143-8fe9-80f7bae2f943-tls-certs\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.413616 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.413520    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ffm4\" (UniqueName: \"kubernetes.io/projected/c228df05-af0d-4143-8fe9-80f7bae2f943-kube-api-access-6ffm4\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.413825 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.413791    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-home\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.413886 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.413558    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-tmp-dir\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.413971 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.413886    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-model-cache\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.414215 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.414189    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-tmp-dir\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.416053 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.416024    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-dshm\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.416176 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.416159    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c228df05-af0d-4143-8fe9-80f7bae2f943-tls-certs\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.421909 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.421860    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ffm4\" (UniqueName: \"kubernetes.io/projected/c228df05-af0d-4143-8fe9-80f7bae2f943-kube-api-access-6ffm4\") pod \"rate-test-model-91cb9f-kserve-74b4578c95-t2xmm\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") " pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.619259 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.619166    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:09.765398 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:09.765106    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"]
Jul 06 18:23:10.713417 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:10.713380    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm" event={"ID":"c228df05-af0d-4143-8fe9-80f7bae2f943","Type":"ContainerStarted","Data":"e4a26bc7d8bfb9c54157513c2f043ad4b4abae006b5c3ca18190af0dbeae5b7e"}
Jul 06 18:23:10.713838 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:10.713432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm" event={"ID":"c228df05-af0d-4143-8fe9-80f7bae2f943","Type":"ContainerStarted","Data":"36a102ea6fd293194a90d5640b1bd10ac0a1e5385e83f10c85f24d6234e68269"}
Jul 06 18:23:10.730637 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:10.730574    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm" podStartSLOduration=1.730553864 podStartE2EDuration="1.730553864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:23:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:23:10.729039868 +0000 UTC m=+2566.322494426" watchObservedRunningTime="2026-07-06 18:23:10.730553864 +0000 UTC m=+2566.324008393"
Jul 06 18:23:19.620079 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:19.620038    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:19.620622 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:19.620115    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:19.633034 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:19.633001    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:19.775785 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:19.775756    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:23:34.685335 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.685299    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"]
Jul 06 18:23:34.689643 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.689615    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.692316 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.692292    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2861c1\"/\"default-dockercfg-pvs5b\""
Jul 06 18:23:34.693615 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.693589    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2861c1\"/\"kube-root-ca.crt\""
Jul 06 18:23:34.693781 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.693756    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2861c1\"/\"rate-test-model-2861c1-kserve-self-signed-certs\""
Jul 06 18:23:34.693989 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.693849    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2861c1\"/\"openshift-service-ca.crt\""
Jul 06 18:23:34.696404 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.696382    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"]
Jul 06 18:23:34.731348 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.731312    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2ad041a-d70d-4376-9454-aef7796ca705-tls-certs\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.731550 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.731370    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-model-cache\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.731550 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.731438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-dshm\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.731643 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.731543    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-tmp-dir\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.731643 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.731581    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-home\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.731643 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.731624    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m58cl\" (UniqueName: \"kubernetes.io/projected/d2ad041a-d70d-4376-9454-aef7796ca705-kube-api-access-m58cl\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.832358 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.832312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-home\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.832580 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.832369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m58cl\" (UniqueName: \"kubernetes.io/projected/d2ad041a-d70d-4376-9454-aef7796ca705-kube-api-access-m58cl\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.832580 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.832400    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2ad041a-d70d-4376-9454-aef7796ca705-tls-certs\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.832580 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.832453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-model-cache\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.832580 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.832479    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-dshm\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.832580 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.832538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-tmp-dir\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.832853 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.832780    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-home\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.832971 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.832949    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-model-cache\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.833033 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.832998    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-tmp-dir\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.834866 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.834843    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-dshm\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.835347 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.835326    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2ad041a-d70d-4376-9454-aef7796ca705-tls-certs\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:34.840054 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:34.840025    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m58cl\" (UniqueName: \"kubernetes.io/projected/d2ad041a-d70d-4376-9454-aef7796ca705-kube-api-access-m58cl\") pod \"rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") " pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:35.002315 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:35.002269    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:35.143997 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:35.143969    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"]
Jul 06 18:23:35.146676 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:23:35.146640    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2ad041a_d70d_4376_9454_aef7796ca705.slice/crio-8c2b14b9bffa4c8ee1969af8ee1312af8b145768a0a662ec696817bfb2c0975f WatchSource:0}: Error finding container 8c2b14b9bffa4c8ee1969af8ee1312af8b145768a0a662ec696817bfb2c0975f: Status 404 returned error can't find the container with id 8c2b14b9bffa4c8ee1969af8ee1312af8b145768a0a662ec696817bfb2c0975f
Jul 06 18:23:35.845211 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:35.845177    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x" event={"ID":"d2ad041a-d70d-4376-9454-aef7796ca705","Type":"ContainerStarted","Data":"2d54713f7feab0b1ffdff602134492d199072737d8f569a9fcfdf92b5913a073"}
Jul 06 18:23:35.845692 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:35.845221    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x" event={"ID":"d2ad041a-d70d-4376-9454-aef7796ca705","Type":"ContainerStarted","Data":"8c2b14b9bffa4c8ee1969af8ee1312af8b145768a0a662ec696817bfb2c0975f"}
Jul 06 18:23:35.862773 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:35.862713    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x" podStartSLOduration=1.862693927 podStartE2EDuration="1.862693927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:23:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:23:35.86058292 +0000 UTC m=+2591.454037450" watchObservedRunningTime="2026-07-06 18:23:35.862693927 +0000 UTC m=+2591.456148457"
Jul 06 18:23:45.003231 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:45.003194    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:45.003695 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:45.003459    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:45.016033 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:45.016007    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:23:45.903802 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:23:45.903764    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:24:32.479863 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:32.479822    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"]
Jul 06 18:24:33.399637 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:33.399600    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"]
Jul 06 18:24:35.085543 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:35.085499    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"]
Jul 06 18:24:35.086008 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:35.085773    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh" podUID="28ea67d5-aeb2-4fca-a67a-f652da9ce31c" containerName="maas-api" containerID="cri-o://c3a707e3703e0c77a968e5a40872837804989b743d68d809eb4d9ade935d580c" gracePeriod=30
Jul 06 18:24:35.552854 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:35.552820    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:24:35.691670 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:35.691640    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7l9z\" (UniqueName: \"kubernetes.io/projected/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-kube-api-access-g7l9z\") pod \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\" (UID: \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\") "
Jul 06 18:24:35.691887 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:35.691719    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-openshift-service-ca\") pod \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\" (UID: \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\") "
Jul 06 18:24:35.691887 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:35.691788    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-maas-api-tls\") pod \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\" (UID: \"28ea67d5-aeb2-4fca-a67a-f652da9ce31c\") "
Jul 06 18:24:35.692132 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:35.692106    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28ea67d5-aeb2-4fca-a67a-f652da9ce31c" (UID: "28ea67d5-aeb2-4fca-a67a-f652da9ce31c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:24:35.693967 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:35.693940    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28ea67d5-aeb2-4fca-a67a-f652da9ce31c" (UID: "28ea67d5-aeb2-4fca-a67a-f652da9ce31c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:24:35.693967 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:35.693955    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-kube-api-access-g7l9z" (OuterVolumeSpecName: "kube-api-access-g7l9z") pod "28ea67d5-aeb2-4fca-a67a-f652da9ce31c" (UID: "28ea67d5-aeb2-4fca-a67a-f652da9ce31c"). InnerVolumeSpecName "kube-api-access-g7l9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:24:35.792883 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:35.792843    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7l9z\" (UniqueName: \"kubernetes.io/projected/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-kube-api-access-g7l9z\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:35.792883 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:35.792874    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-openshift-service-ca\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:35.792883 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:35.792887    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28ea67d5-aeb2-4fca-a67a-f652da9ce31c-maas-api-tls\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:36.156015 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:36.155915    2581 generic.go:356] "Generic (PLEG): container finished" podID="28ea67d5-aeb2-4fca-a67a-f652da9ce31c" containerID="c3a707e3703e0c77a968e5a40872837804989b743d68d809eb4d9ade935d580c" exitCode=0
Jul 06 18:24:36.156015 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:36.155971    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"
Jul 06 18:24:36.156015 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:36.155990    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh" event={"ID":"28ea67d5-aeb2-4fca-a67a-f652da9ce31c","Type":"ContainerDied","Data":"c3a707e3703e0c77a968e5a40872837804989b743d68d809eb4d9ade935d580c"}
Jul 06 18:24:36.156576 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:36.156057    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh" event={"ID":"28ea67d5-aeb2-4fca-a67a-f652da9ce31c","Type":"ContainerDied","Data":"53520152e2a1834714e0c53ee82134bd1a7039a62e365ac1f1b34a7a49d63775"}
Jul 06 18:24:36.156576 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:36.156087    2581 scope.go:117] "RemoveContainer" containerID="c3a707e3703e0c77a968e5a40872837804989b743d68d809eb4d9ade935d580c"
Jul 06 18:24:36.165592 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:36.165571    2581 scope.go:117] "RemoveContainer" containerID="c3a707e3703e0c77a968e5a40872837804989b743d68d809eb4d9ade935d580c"
Jul 06 18:24:36.165939 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:24:36.165885    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3a707e3703e0c77a968e5a40872837804989b743d68d809eb4d9ade935d580c\": container with ID starting with c3a707e3703e0c77a968e5a40872837804989b743d68d809eb4d9ade935d580c not found: ID does not exist" containerID="c3a707e3703e0c77a968e5a40872837804989b743d68d809eb4d9ade935d580c"
Jul 06 18:24:36.166039 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:36.165947    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3a707e3703e0c77a968e5a40872837804989b743d68d809eb4d9ade935d580c"} err="failed to get container status \"c3a707e3703e0c77a968e5a40872837804989b743d68d809eb4d9ade935d580c\": rpc error: code = NotFound desc = could not find container \"c3a707e3703e0c77a968e5a40872837804989b743d68d809eb4d9ade935d580c\": container with ID starting with c3a707e3703e0c77a968e5a40872837804989b743d68d809eb4d9ade935d580c not found: ID does not exist"
Jul 06 18:24:36.177512 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:36.177482    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"]
Jul 06 18:24:36.181252 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:36.181223    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-91cb9f-5989bbdb8d-nq7qh"]
Jul 06 18:24:36.912661 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:36.912631    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28ea67d5-aeb2-4fca-a67a-f652da9ce31c" path="/var/lib/kubelet/pods/28ea67d5-aeb2-4fca-a67a-f652da9ce31c/volumes"
Jul 06 18:24:47.501006 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.500937    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm" podUID="c228df05-af0d-4143-8fe9-80f7bae2f943" containerName="main" containerID="cri-o://e4a26bc7d8bfb9c54157513c2f043ad4b4abae006b5c3ca18190af0dbeae5b7e" gracePeriod=45
Jul 06 18:24:47.767917 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.767820    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:24:47.905306 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.905265    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c228df05-af0d-4143-8fe9-80f7bae2f943-tls-certs\") pod \"c228df05-af0d-4143-8fe9-80f7bae2f943\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") "
Jul 06 18:24:47.905501 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.905349    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ffm4\" (UniqueName: \"kubernetes.io/projected/c228df05-af0d-4143-8fe9-80f7bae2f943-kube-api-access-6ffm4\") pod \"c228df05-af0d-4143-8fe9-80f7bae2f943\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") "
Jul 06 18:24:47.905501 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.905457    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-home\") pod \"c228df05-af0d-4143-8fe9-80f7bae2f943\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") "
Jul 06 18:24:47.905676 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.905505    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-dshm\") pod \"c228df05-af0d-4143-8fe9-80f7bae2f943\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") "
Jul 06 18:24:47.905676 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.905527    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-model-cache\") pod \"c228df05-af0d-4143-8fe9-80f7bae2f943\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") "
Jul 06 18:24:47.905676 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.905606    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-tmp-dir\") pod \"c228df05-af0d-4143-8fe9-80f7bae2f943\" (UID: \"c228df05-af0d-4143-8fe9-80f7bae2f943\") "
Jul 06 18:24:47.905842 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.905757    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-home" (OuterVolumeSpecName: "home") pod "c228df05-af0d-4143-8fe9-80f7bae2f943" (UID: "c228df05-af0d-4143-8fe9-80f7bae2f943"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:24:47.905842 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.905824    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-model-cache" (OuterVolumeSpecName: "model-cache") pod "c228df05-af0d-4143-8fe9-80f7bae2f943" (UID: "c228df05-af0d-4143-8fe9-80f7bae2f943"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:24:47.905988 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.905950    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-home\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:47.905988 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.905971    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-model-cache\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:47.906086 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.906070    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c228df05-af0d-4143-8fe9-80f7bae2f943" (UID: "c228df05-af0d-4143-8fe9-80f7bae2f943"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:24:47.907561 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.907534    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c228df05-af0d-4143-8fe9-80f7bae2f943-kube-api-access-6ffm4" (OuterVolumeSpecName: "kube-api-access-6ffm4") pod "c228df05-af0d-4143-8fe9-80f7bae2f943" (UID: "c228df05-af0d-4143-8fe9-80f7bae2f943"). InnerVolumeSpecName "kube-api-access-6ffm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:24:47.907946 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.907923    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-dshm" (OuterVolumeSpecName: "dshm") pod "c228df05-af0d-4143-8fe9-80f7bae2f943" (UID: "c228df05-af0d-4143-8fe9-80f7bae2f943"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:24:47.908015 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:47.907967    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c228df05-af0d-4143-8fe9-80f7bae2f943-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c228df05-af0d-4143-8fe9-80f7bae2f943" (UID: "c228df05-af0d-4143-8fe9-80f7bae2f943"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:24:48.006682 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.006639    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-tmp-dir\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:48.006682 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.006675    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c228df05-af0d-4143-8fe9-80f7bae2f943-tls-certs\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:48.006682 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.006684    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ffm4\" (UniqueName: \"kubernetes.io/projected/c228df05-af0d-4143-8fe9-80f7bae2f943-kube-api-access-6ffm4\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:48.006682 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.006693    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c228df05-af0d-4143-8fe9-80f7bae2f943-dshm\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:48.223962 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.223917    2581 generic.go:356] "Generic (PLEG): container finished" podID="c228df05-af0d-4143-8fe9-80f7bae2f943" containerID="e4a26bc7d8bfb9c54157513c2f043ad4b4abae006b5c3ca18190af0dbeae5b7e" exitCode=0
Jul 06 18:24:48.223962 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.223959    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"
Jul 06 18:24:48.224216 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.224002    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm" event={"ID":"c228df05-af0d-4143-8fe9-80f7bae2f943","Type":"ContainerDied","Data":"e4a26bc7d8bfb9c54157513c2f043ad4b4abae006b5c3ca18190af0dbeae5b7e"}
Jul 06 18:24:48.224216 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.224050    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm" event={"ID":"c228df05-af0d-4143-8fe9-80f7bae2f943","Type":"ContainerDied","Data":"36a102ea6fd293194a90d5640b1bd10ac0a1e5385e83f10c85f24d6234e68269"}
Jul 06 18:24:48.224216 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.224076    2581 scope.go:117] "RemoveContainer" containerID="e4a26bc7d8bfb9c54157513c2f043ad4b4abae006b5c3ca18190af0dbeae5b7e"
Jul 06 18:24:48.233643 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.233622    2581 scope.go:117] "RemoveContainer" containerID="e4a26bc7d8bfb9c54157513c2f043ad4b4abae006b5c3ca18190af0dbeae5b7e"
Jul 06 18:24:48.233996 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:24:48.233970    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4a26bc7d8bfb9c54157513c2f043ad4b4abae006b5c3ca18190af0dbeae5b7e\": container with ID starting with e4a26bc7d8bfb9c54157513c2f043ad4b4abae006b5c3ca18190af0dbeae5b7e not found: ID does not exist" containerID="e4a26bc7d8bfb9c54157513c2f043ad4b4abae006b5c3ca18190af0dbeae5b7e"
Jul 06 18:24:48.234101 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.234004    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4a26bc7d8bfb9c54157513c2f043ad4b4abae006b5c3ca18190af0dbeae5b7e"} err="failed to get container status \"e4a26bc7d8bfb9c54157513c2f043ad4b4abae006b5c3ca18190af0dbeae5b7e\": rpc error: code = NotFound desc = could not find container \"e4a26bc7d8bfb9c54157513c2f043ad4b4abae006b5c3ca18190af0dbeae5b7e\": container with ID starting with e4a26bc7d8bfb9c54157513c2f043ad4b4abae006b5c3ca18190af0dbeae5b7e not found: ID does not exist"
Jul 06 18:24:48.244601 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.244565    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"]
Jul 06 18:24:48.249833 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.249800    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-91cb9f/rate-test-model-91cb9f-kserve-74b4578c95-t2xmm"]
Jul 06 18:24:48.420445 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.420404    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x" podUID="d2ad041a-d70d-4376-9454-aef7796ca705" containerName="main" containerID="cri-o://2d54713f7feab0b1ffdff602134492d199072737d8f569a9fcfdf92b5913a073" gracePeriod=45
Jul 06 18:24:48.685017 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.684984    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:24:48.813747 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.813658    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m58cl\" (UniqueName: \"kubernetes.io/projected/d2ad041a-d70d-4376-9454-aef7796ca705-kube-api-access-m58cl\") pod \"d2ad041a-d70d-4376-9454-aef7796ca705\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") "
Jul 06 18:24:48.813747 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.813704    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-dshm\") pod \"d2ad041a-d70d-4376-9454-aef7796ca705\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") "
Jul 06 18:24:48.813992 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.813758    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-home\") pod \"d2ad041a-d70d-4376-9454-aef7796ca705\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") "
Jul 06 18:24:48.813992 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.813782    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-model-cache\") pod \"d2ad041a-d70d-4376-9454-aef7796ca705\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") "
Jul 06 18:24:48.813992 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.813835    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2ad041a-d70d-4376-9454-aef7796ca705-tls-certs\") pod \"d2ad041a-d70d-4376-9454-aef7796ca705\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") "
Jul 06 18:24:48.813992 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.813953    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-tmp-dir\") pod \"d2ad041a-d70d-4376-9454-aef7796ca705\" (UID: \"d2ad041a-d70d-4376-9454-aef7796ca705\") "
Jul 06 18:24:48.814205 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.814084    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-model-cache" (OuterVolumeSpecName: "model-cache") pod "d2ad041a-d70d-4376-9454-aef7796ca705" (UID: "d2ad041a-d70d-4376-9454-aef7796ca705"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:24:48.814205 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.814103    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-home" (OuterVolumeSpecName: "home") pod "d2ad041a-d70d-4376-9454-aef7796ca705" (UID: "d2ad041a-d70d-4376-9454-aef7796ca705"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:24:48.814325 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.814230    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d2ad041a-d70d-4376-9454-aef7796ca705" (UID: "d2ad041a-d70d-4376-9454-aef7796ca705"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:24:48.814325 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.814241    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-home\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:48.814325 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.814285    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-model-cache\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:48.816077 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.816044    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-dshm" (OuterVolumeSpecName: "dshm") pod "d2ad041a-d70d-4376-9454-aef7796ca705" (UID: "d2ad041a-d70d-4376-9454-aef7796ca705"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:24:48.816077 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.816057    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2ad041a-d70d-4376-9454-aef7796ca705-kube-api-access-m58cl" (OuterVolumeSpecName: "kube-api-access-m58cl") pod "d2ad041a-d70d-4376-9454-aef7796ca705" (UID: "d2ad041a-d70d-4376-9454-aef7796ca705"). InnerVolumeSpecName "kube-api-access-m58cl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:24:48.816201 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.816128    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ad041a-d70d-4376-9454-aef7796ca705-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d2ad041a-d70d-4376-9454-aef7796ca705" (UID: "d2ad041a-d70d-4376-9454-aef7796ca705"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:24:48.912866 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.912833    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c228df05-af0d-4143-8fe9-80f7bae2f943" path="/var/lib/kubelet/pods/c228df05-af0d-4143-8fe9-80f7bae2f943/volumes"
Jul 06 18:24:48.915502 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.915478    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m58cl\" (UniqueName: \"kubernetes.io/projected/d2ad041a-d70d-4376-9454-aef7796ca705-kube-api-access-m58cl\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:48.915579 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.915504    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-dshm\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:48.915579 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.915515    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2ad041a-d70d-4376-9454-aef7796ca705-tls-certs\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:48.915579 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:48.915523    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2ad041a-d70d-4376-9454-aef7796ca705-tmp-dir\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:24:49.230641 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:49.230609    2581 generic.go:356] "Generic (PLEG): container finished" podID="d2ad041a-d70d-4376-9454-aef7796ca705" containerID="2d54713f7feab0b1ffdff602134492d199072737d8f569a9fcfdf92b5913a073" exitCode=0
Jul 06 18:24:49.230854 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:49.230650    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"
Jul 06 18:24:49.230854 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:49.230708    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x" event={"ID":"d2ad041a-d70d-4376-9454-aef7796ca705","Type":"ContainerDied","Data":"2d54713f7feab0b1ffdff602134492d199072737d8f569a9fcfdf92b5913a073"}
Jul 06 18:24:49.230854 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:49.230762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x" event={"ID":"d2ad041a-d70d-4376-9454-aef7796ca705","Type":"ContainerDied","Data":"8c2b14b9bffa4c8ee1969af8ee1312af8b145768a0a662ec696817bfb2c0975f"}
Jul 06 18:24:49.230854 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:49.230786    2581 scope.go:117] "RemoveContainer" containerID="2d54713f7feab0b1ffdff602134492d199072737d8f569a9fcfdf92b5913a073"
Jul 06 18:24:49.240642 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:49.240621    2581 scope.go:117] "RemoveContainer" containerID="2d54713f7feab0b1ffdff602134492d199072737d8f569a9fcfdf92b5913a073"
Jul 06 18:24:49.240980 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:24:49.240960    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d54713f7feab0b1ffdff602134492d199072737d8f569a9fcfdf92b5913a073\": container with ID starting with 2d54713f7feab0b1ffdff602134492d199072737d8f569a9fcfdf92b5913a073 not found: ID does not exist" containerID="2d54713f7feab0b1ffdff602134492d199072737d8f569a9fcfdf92b5913a073"
Jul 06 18:24:49.241071 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:49.240992    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d54713f7feab0b1ffdff602134492d199072737d8f569a9fcfdf92b5913a073"} err="failed to get container status \"2d54713f7feab0b1ffdff602134492d199072737d8f569a9fcfdf92b5913a073\": rpc error: code = NotFound desc = could not find container \"2d54713f7feab0b1ffdff602134492d199072737d8f569a9fcfdf92b5913a073\": container with ID starting with 2d54713f7feab0b1ffdff602134492d199072737d8f569a9fcfdf92b5913a073 not found: ID does not exist"
Jul 06 18:24:49.247151 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:49.247121    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"]
Jul 06 18:24:49.248882 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:49.248855    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2861c1/rate-test-model-2861c1-kserve-65dd9bccbc-ftz4x"]
Jul 06 18:24:50.912501 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:24:50.912460    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2ad041a-d70d-4376-9454-aef7796ca705" path="/var/lib/kubelet/pods/d2ad041a-d70d-4376-9454-aef7796ca705/volumes"
Jul 06 18:25:02.107521 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:02.107473    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"]
Jul 06 18:25:02.108416 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:02.107803    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh" podUID="b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e" containerName="maas-api" containerID="cri-o://7e8c96825e8f8a96a681eace55dd333a78e00ef28dd5dfdc833040a866a1b7ad" gracePeriod=30
Jul 06 18:25:02.570366 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:02.570334    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:25:02.627978 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:02.627932    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdfk9\" (UniqueName: \"kubernetes.io/projected/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-kube-api-access-pdfk9\") pod \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\" (UID: \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\") "
Jul 06 18:25:02.628170 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:02.627990    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-openshift-service-ca\") pod \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\" (UID: \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\") "
Jul 06 18:25:02.628170 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:02.628067    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-maas-api-tls\") pod \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\" (UID: \"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e\") "
Jul 06 18:25:02.628358 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:02.628331    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e" (UID: "b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:25:02.630211 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:02.630177    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-kube-api-access-pdfk9" (OuterVolumeSpecName: "kube-api-access-pdfk9") pod "b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e" (UID: "b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e"). InnerVolumeSpecName "kube-api-access-pdfk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:25:02.630349 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:02.630251    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e" (UID: "b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:25:02.729551 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:02.729509    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-maas-api-tls\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:25:02.729551 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:02.729544    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdfk9\" (UniqueName: \"kubernetes.io/projected/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-kube-api-access-pdfk9\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:25:02.729551 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:02.729556    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e-openshift-service-ca\") on node \"ip-10-0-133-236.ec2.internal\" DevicePath \"\""
Jul 06 18:25:03.309025 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:03.308981    2581 generic.go:356] "Generic (PLEG): container finished" podID="b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e" containerID="7e8c96825e8f8a96a681eace55dd333a78e00ef28dd5dfdc833040a866a1b7ad" exitCode=0
Jul 06 18:25:03.309025 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:03.309027    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"
Jul 06 18:25:03.309518 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:03.309066    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh" event={"ID":"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e","Type":"ContainerDied","Data":"7e8c96825e8f8a96a681eace55dd333a78e00ef28dd5dfdc833040a866a1b7ad"}
Jul 06 18:25:03.309518 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:03.309121    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh" event={"ID":"b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e","Type":"ContainerDied","Data":"116e39a5432989340827836f82c5eed806564d9f55015421280f7287e31a3fd7"}
Jul 06 18:25:03.309518 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:03.309143    2581 scope.go:117] "RemoveContainer" containerID="7e8c96825e8f8a96a681eace55dd333a78e00ef28dd5dfdc833040a866a1b7ad"
Jul 06 18:25:03.318400 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:03.318376    2581 scope.go:117] "RemoveContainer" containerID="7e8c96825e8f8a96a681eace55dd333a78e00ef28dd5dfdc833040a866a1b7ad"
Jul 06 18:25:03.318706 ip-10-0-133-236 kubenswrapper[2581]: E0706 18:25:03.318689    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e8c96825e8f8a96a681eace55dd333a78e00ef28dd5dfdc833040a866a1b7ad\": container with ID starting with 7e8c96825e8f8a96a681eace55dd333a78e00ef28dd5dfdc833040a866a1b7ad not found: ID does not exist" containerID="7e8c96825e8f8a96a681eace55dd333a78e00ef28dd5dfdc833040a866a1b7ad"
Jul 06 18:25:03.318764 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:03.318716    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e8c96825e8f8a96a681eace55dd333a78e00ef28dd5dfdc833040a866a1b7ad"} err="failed to get container status \"7e8c96825e8f8a96a681eace55dd333a78e00ef28dd5dfdc833040a866a1b7ad\": rpc error: code = NotFound desc = could not find container \"7e8c96825e8f8a96a681eace55dd333a78e00ef28dd5dfdc833040a866a1b7ad\": container with ID starting with 7e8c96825e8f8a96a681eace55dd333a78e00ef28dd5dfdc833040a866a1b7ad not found: ID does not exist"
Jul 06 18:25:03.326410 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:03.326374    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"]
Jul 06 18:25:03.328766 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:03.328736    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-2861c1-76b6468d8-qncbh"]
Jul 06 18:25:04.912458 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:04.912420    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e" path="/var/lib/kubelet/pods/b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e/volumes"
Jul 06 18:25:08.615289 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:08.615248    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58fbc99c78-q8tnh_6d58e0a4-6519-4402-a46d-246c4068c373/authorino/0.log"
Jul 06 18:25:13.133430 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:13.133398    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2ngwb_1e76a1ea-7310-49a7-a735-6556f2add542/manager/0.log"
Jul 06 18:25:13.250431 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:13.250392    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-f42lr_0d6517d6-c2c5-49ca-abce-dfc7a435deeb/manager/0.log"
Jul 06 18:25:13.368492 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:13.368458    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5b645c74c6-wr4mv_f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597/maas-api/0.log"
Jul 06 18:25:13.478715 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:13.478683    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722680-n5x7t_1c6a6f90-ede8-4f61-b024-602381ddcf3f/cleanup/0.log"
Jul 06 18:25:13.589357 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:13.589332    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722695-47mn6_46d55e05-a713-48ea-966c-c7210d1ff7c6/cleanup/0.log"
Jul 06 18:25:13.709760 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:13.709711    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-857d875667-jtqxt_37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3/manager/0.log"
Jul 06 18:25:13.956517 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:13.956473    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kq5r8_16e03d15-556c-4eaf-8f49-e5210b9422bc/manager/0.log"
Jul 06 18:25:14.189405 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:14.189375    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74668f54-rpscx_5389f834-f90e-4388-b2df-fbf5ea3ffe55/copy-manifests/0.log"
Jul 06 18:25:14.196198 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:14.196173    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74668f54-rpscx_5389f834-f90e-4388-b2df-fbf5ea3ffe55/manager/0.log"
Jul 06 18:25:14.433968 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:14.433858    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-ft2sq_338ff4a4-4c2b-42ef-9bd8-92e534da391d/postgres/0.log"
Jul 06 18:25:15.721035 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:15.720999    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58fbc99c78-q8tnh_6d58e0a4-6519-4402-a46d-246c4068c373/authorino/0.log"
Jul 06 18:25:15.839241 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:15.839204    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9t5th_6efca74b-5e60-4f88-8308-7cdac638cc62/manager/0.log"
Jul 06 18:25:15.951689 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:15.951662    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-vdf5l_9b00adb8-3ac5-4dad-bc77-bd4ae0cad169/manager/0.log"
Jul 06 18:25:16.179777 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:16.179679    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2wnrn_e6b10695-d2ce-417d-8d5f-32ebb6ed3223/registry-server/0.log"
Jul 06 18:25:16.305103 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:16.305065    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qzgh6_41a5db61-c70f-4b34-8e12-2200729b3d36/manager/0.log"
Jul 06 18:25:16.534821 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:16.534786    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-5824s_c46eb0f3-b4c3-44b0-af07-9eb55481a17e/manager/0.log"
Jul 06 18:25:17.214122 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:17.214089    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-98dc6956-pmkj5_92359562-7966-400d-87ed-d7d35b885d40/kube-auth-proxy/0.log"
Jul 06 18:25:17.455178 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:17.455146    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-zr75s_f15e0340-5a61-44aa-b651-331f8a5c32fe/payload-pre-processing/0.log"
Jul 06 18:25:17.572567 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:17.572479    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-skm4k_89d6393c-dc4c-4cf1-bfaa-b389cc6fb439/payload-processing/0.log"
Jul 06 18:25:18.355685 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:18.355648    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5l78w_8b1c3d0f-a6f7-4ac6-900c-93ef80e7a632/main/0.log"
Jul 06 18:25:22.913294 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.913260    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q4mx2/must-gather-wcfc6"]
Jul 06 18:25:22.913800 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.913728    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e" containerName="maas-api"
Jul 06 18:25:22.913800 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.913746    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e" containerName="maas-api"
Jul 06 18:25:22.913800 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.913763    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28ea67d5-aeb2-4fca-a67a-f652da9ce31c" containerName="maas-api"
Jul 06 18:25:22.913800 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.913771    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="28ea67d5-aeb2-4fca-a67a-f652da9ce31c" containerName="maas-api"
Jul 06 18:25:22.914049 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.913802    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c228df05-af0d-4143-8fe9-80f7bae2f943" containerName="main"
Jul 06 18:25:22.914049 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.913810    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c228df05-af0d-4143-8fe9-80f7bae2f943" containerName="main"
Jul 06 18:25:22.914049 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.913822    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2ad041a-d70d-4376-9454-aef7796ca705" containerName="main"
Jul 06 18:25:22.914049 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.913830    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2ad041a-d70d-4376-9454-aef7796ca705" containerName="main"
Jul 06 18:25:22.914049 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.913935    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2ad041a-d70d-4376-9454-aef7796ca705" containerName="main"
Jul 06 18:25:22.914049 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.913947    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9d1b5e3-f61f-4f85-98f2-2f9f83953f1e" containerName="maas-api"
Jul 06 18:25:22.914049 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.913958    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="28ea67d5-aeb2-4fca-a67a-f652da9ce31c" containerName="maas-api"
Jul 06 18:25:22.914049 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.913969    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c228df05-af0d-4143-8fe9-80f7bae2f943" containerName="main"
Jul 06 18:25:22.919120 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.919092    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q4mx2/must-gather-wcfc6"
Jul 06 18:25:22.921995 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.921972    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-q4mx2\"/\"default-dockercfg-qlwpk\""
Jul 06 18:25:22.922161 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.922037    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q4mx2\"/\"kube-root-ca.crt\""
Jul 06 18:25:22.923105 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.923077    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q4mx2\"/\"openshift-service-ca.crt\""
Jul 06 18:25:22.923719 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.923697    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q4mx2/must-gather-wcfc6"]
Jul 06 18:25:22.992198 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.992155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e8b255ae-9773-45c2-8281-32eb8ee4d284-must-gather-output\") pod \"must-gather-wcfc6\" (UID: \"e8b255ae-9773-45c2-8281-32eb8ee4d284\") " pod="openshift-must-gather-q4mx2/must-gather-wcfc6"
Jul 06 18:25:22.992366 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:22.992223    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpjs8\" (UniqueName: \"kubernetes.io/projected/e8b255ae-9773-45c2-8281-32eb8ee4d284-kube-api-access-tpjs8\") pod \"must-gather-wcfc6\" (UID: \"e8b255ae-9773-45c2-8281-32eb8ee4d284\") " pod="openshift-must-gather-q4mx2/must-gather-wcfc6"
Jul 06 18:25:23.093465 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:23.093423    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e8b255ae-9773-45c2-8281-32eb8ee4d284-must-gather-output\") pod \"must-gather-wcfc6\" (UID: \"e8b255ae-9773-45c2-8281-32eb8ee4d284\") " pod="openshift-must-gather-q4mx2/must-gather-wcfc6"
Jul 06 18:25:23.093631 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:23.093476    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpjs8\" (UniqueName: \"kubernetes.io/projected/e8b255ae-9773-45c2-8281-32eb8ee4d284-kube-api-access-tpjs8\") pod \"must-gather-wcfc6\" (UID: \"e8b255ae-9773-45c2-8281-32eb8ee4d284\") " pod="openshift-must-gather-q4mx2/must-gather-wcfc6"
Jul 06 18:25:23.093855 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:23.093830    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e8b255ae-9773-45c2-8281-32eb8ee4d284-must-gather-output\") pod \"must-gather-wcfc6\" (UID: \"e8b255ae-9773-45c2-8281-32eb8ee4d284\") " pod="openshift-must-gather-q4mx2/must-gather-wcfc6"
Jul 06 18:25:23.101339 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:23.101304    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpjs8\" (UniqueName: \"kubernetes.io/projected/e8b255ae-9773-45c2-8281-32eb8ee4d284-kube-api-access-tpjs8\") pod \"must-gather-wcfc6\" (UID: \"e8b255ae-9773-45c2-8281-32eb8ee4d284\") " pod="openshift-must-gather-q4mx2/must-gather-wcfc6"
Jul 06 18:25:23.230084 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:23.230044    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q4mx2/must-gather-wcfc6"
Jul 06 18:25:23.376121 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:23.376093    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q4mx2/must-gather-wcfc6"]
Jul 06 18:25:23.378154 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:25:23.378111    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8b255ae_9773_45c2_8281_32eb8ee4d284.slice/crio-d637f77deab830db95520ffdd69ab37cc3953510721d97d8334100f925a5293a WatchSource:0}: Error finding container d637f77deab830db95520ffdd69ab37cc3953510721d97d8334100f925a5293a: Status 404 returned error can't find the container with id d637f77deab830db95520ffdd69ab37cc3953510721d97d8334100f925a5293a
Jul 06 18:25:23.380064 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:23.380041    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 18:25:23.412122 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:23.412085    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q4mx2/must-gather-wcfc6" event={"ID":"e8b255ae-9773-45c2-8281-32eb8ee4d284","Type":"ContainerStarted","Data":"d637f77deab830db95520ffdd69ab37cc3953510721d97d8334100f925a5293a"}
Jul 06 18:25:24.421160 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:24.420610    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q4mx2/must-gather-wcfc6" event={"ID":"e8b255ae-9773-45c2-8281-32eb8ee4d284","Type":"ContainerStarted","Data":"d09b12a7d15fc8f9337fbd44f386e781566bbd2c9f21e351ff17496a7fb20181"}
Jul 06 18:25:24.421160 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:24.420672    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q4mx2/must-gather-wcfc6" event={"ID":"e8b255ae-9773-45c2-8281-32eb8ee4d284","Type":"ContainerStarted","Data":"c2e2c716b7a1ef6421f5c77ca053503ccac392377558cca9dc29c1118bdb4552"}
Jul 06 18:25:24.435655 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:24.435584    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-q4mx2/must-gather-wcfc6" podStartSLOduration=1.688966378 podStartE2EDuration="2.435564427s" totalImagesPullingTime="746.598049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:25:24.434118562 +0000 UTC m=+2700.027573093" watchObservedRunningTime="2026-07-06 18:25:24.435564427 +0000 UTC m=+2700.029018973"
Jul 06 18:25:25.750446 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:25.750408    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vpff9_d44eac64-207b-4b14-b10b-9d54cd3ae49b/global-pull-secret-syncer/0.log"
Jul 06 18:25:25.918747 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:25.918697    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dzlq8_9b00296d-a49b-4174-b08f-191b31d4c7a5/konnectivity-agent/0.log"
Jul 06 18:25:26.004178 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:26.004064    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-236.ec2.internal_f393a703fb7ce0f536ef1eee45ebaae3/haproxy/0.log"
Jul 06 18:25:31.596917 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:31.596250    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58fbc99c78-q8tnh_6d58e0a4-6519-4402-a46d-246c4068c373/authorino/0.log"
Jul 06 18:25:31.637769 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:31.637731    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9t5th_6efca74b-5e60-4f88-8308-7cdac638cc62/manager/0.log"
Jul 06 18:25:31.675960 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:31.675919    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-vdf5l_9b00adb8-3ac5-4dad-bc77-bd4ae0cad169/manager/0.log"
Jul 06 18:25:31.759861 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:31.759802    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2wnrn_e6b10695-d2ce-417d-8d5f-32ebb6ed3223/registry-server/0.log"
Jul 06 18:25:31.902428 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:31.902331    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qzgh6_41a5db61-c70f-4b34-8e12-2200729b3d36/manager/0.log"
Jul 06 18:25:32.048921 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:32.048158    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-5824s_c46eb0f3-b4c3-44b0-af07-9eb55481a17e/manager/0.log"
Jul 06 18:25:34.291336 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:34.291297    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kv695_905c962d-d69e-4b0a-ae61-27c38fdcd15f/node-exporter/0.log"
Jul 06 18:25:34.314740 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:34.314708    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kv695_905c962d-d69e-4b0a-ae61-27c38fdcd15f/kube-rbac-proxy/0.log"
Jul 06 18:25:34.341052 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:34.341023    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kv695_905c962d-d69e-4b0a-ae61-27c38fdcd15f/init-textfile/0.log"
Jul 06 18:25:36.087565 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.087530    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-7lvhg_f3682cbe-9ac5-4915-9e80-8c897e4a3996/networking-console-plugin/0.log"
Jul 06 18:25:36.406886 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.406793    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"]
Jul 06 18:25:36.414806 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.414767    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.421340 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.421303    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"]
Jul 06 18:25:36.551005 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.550963    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c264898f-117b-4e7d-a415-53e7dab76b67-proc\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.551295 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.551019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c264898f-117b-4e7d-a415-53e7dab76b67-sys\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.551295 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.551068    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c264898f-117b-4e7d-a415-53e7dab76b67-podres\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.551295 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.551091    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbxvz\" (UniqueName: \"kubernetes.io/projected/c264898f-117b-4e7d-a415-53e7dab76b67-kube-api-access-wbxvz\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.551295 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.551191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c264898f-117b-4e7d-a415-53e7dab76b67-lib-modules\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.652129 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.652090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c264898f-117b-4e7d-a415-53e7dab76b67-lib-modules\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.652351 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.652148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c264898f-117b-4e7d-a415-53e7dab76b67-proc\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.652351 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.652178    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c264898f-117b-4e7d-a415-53e7dab76b67-sys\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.652351 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.652218    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c264898f-117b-4e7d-a415-53e7dab76b67-podres\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.652351 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.652242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbxvz\" (UniqueName: \"kubernetes.io/projected/c264898f-117b-4e7d-a415-53e7dab76b67-kube-api-access-wbxvz\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.652351 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.652297    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c264898f-117b-4e7d-a415-53e7dab76b67-lib-modules\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.652351 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.652324    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c264898f-117b-4e7d-a415-53e7dab76b67-sys\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.652351 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.652297    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c264898f-117b-4e7d-a415-53e7dab76b67-proc\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.652638 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.652360    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c264898f-117b-4e7d-a415-53e7dab76b67-podres\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.663531 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.663444    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbxvz\" (UniqueName: \"kubernetes.io/projected/c264898f-117b-4e7d-a415-53e7dab76b67-kube-api-access-wbxvz\") pod \"perf-node-gather-daemonset-knsfp\" (UID: \"c264898f-117b-4e7d-a415-53e7dab76b67\") " pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.730833 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.730797    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:36.932475 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:36.932442    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"]
Jul 06 18:25:36.935313 ip-10-0-133-236 kubenswrapper[2581]: W0706 18:25:36.935281    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc264898f_117b_4e7d_a415_53e7dab76b67.slice/crio-d2eeeb2f51b846d18e5d1266ac223b42e275d10ec8021c1759084bae518c58f5 WatchSource:0}: Error finding container d2eeeb2f51b846d18e5d1266ac223b42e275d10ec8021c1759084bae518c58f5: Status 404 returned error can't find the container with id d2eeeb2f51b846d18e5d1266ac223b42e275d10ec8021c1759084bae518c58f5
Jul 06 18:25:37.522820 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:37.522776    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp" event={"ID":"c264898f-117b-4e7d-a415-53e7dab76b67","Type":"ContainerStarted","Data":"e37d57a9d236ec81c9bfeb29397eef41c0db46576534d2e005def3142a3f4c86"}
Jul 06 18:25:37.523283 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:37.522839    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp" event={"ID":"c264898f-117b-4e7d-a415-53e7dab76b67","Type":"ContainerStarted","Data":"d2eeeb2f51b846d18e5d1266ac223b42e275d10ec8021c1759084bae518c58f5"}
Jul 06 18:25:37.523283 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:37.522879    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:37.541335 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:37.541260    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp" podStartSLOduration=1.541231823 podStartE2EDuration="1.541231823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:25:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:25:37.538533486 +0000 UTC m=+2713.131988056" watchObservedRunningTime="2026-07-06 18:25:37.541231823 +0000 UTC m=+2713.134686354"
Jul 06 18:25:38.974773 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:38.974735    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-97mtx_a3fe6180-c3f7-4b6f-bb58-591305c33c45/dns/0.log"
Jul 06 18:25:38.996944 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:38.996907    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-97mtx_a3fe6180-c3f7-4b6f-bb58-591305c33c45/kube-rbac-proxy/0.log"
Jul 06 18:25:39.162614 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:39.162581    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v58n8_a79ce496-9313-4d12-912a-b17d2429b3a6/dns-node-resolver/0.log"
Jul 06 18:25:39.762291 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:39.762253    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5gcxt_ae64e4d4-c8c5-47aa-92d6-3cdb8a2a6edd/node-ca/0.log"
Jul 06 18:25:40.999272 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:40.999235    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-98dc6956-pmkj5_92359562-7966-400d-87ed-d7d35b885d40/kube-auth-proxy/0.log"
Jul 06 18:25:41.127032 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:41.126993    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-zr75s_f15e0340-5a61-44aa-b651-331f8a5c32fe/payload-pre-processing/0.log"
Jul 06 18:25:41.212246 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:41.212215    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-skm4k_89d6393c-dc4c-4cf1-bfaa-b389cc6fb439/payload-processing/0.log"
Jul 06 18:25:41.985473 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:41.985443    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xnt6h_df45d8cf-5c5e-4e51-bc0a-029de6595c53/serve-healthcheck-canary/0.log"
Jul 06 18:25:42.502729 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:42.502698    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7nvq4_f78fad60-12aa-4b66-abce-3d490c40a6c5/kube-rbac-proxy/0.log"
Jul 06 18:25:42.524817 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:42.524784    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7nvq4_f78fad60-12aa-4b66-abce-3d490c40a6c5/exporter/0.log"
Jul 06 18:25:42.547594 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:42.547560    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7nvq4_f78fad60-12aa-4b66-abce-3d490c40a6c5/extractor/0.log"
Jul 06 18:25:43.542721 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:43.542688    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-q4mx2/perf-node-gather-daemonset-knsfp"
Jul 06 18:25:44.797135 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:44.797101    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2ngwb_1e76a1ea-7310-49a7-a735-6556f2add542/manager/0.log"
Jul 06 18:25:45.036409 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:45.036372    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-f42lr_0d6517d6-c2c5-49ca-abce-dfc7a435deeb/manager/0.log"
Jul 06 18:25:45.087334 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:45.087155    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5b645c74c6-wr4mv_f5ae26c9-02d5-42e1-9c8a-ef9f41ce8597/maas-api/0.log"
Jul 06 18:25:45.108598 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:45.108567    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722680-n5x7t_1c6a6f90-ede8-4f61-b024-602381ddcf3f/cleanup/0.log"
Jul 06 18:25:45.138438 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:45.138408    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722695-47mn6_46d55e05-a713-48ea-966c-c7210d1ff7c6/cleanup/0.log"
Jul 06 18:25:45.320309 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:45.320243    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-857d875667-jtqxt_37cb94e1-6be3-4be5-b77f-a9ed4d6fd3e3/manager/0.log"
Jul 06 18:25:45.459191 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:45.459153    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kq5r8_16e03d15-556c-4eaf-8f49-e5210b9422bc/manager/0.log"
Jul 06 18:25:45.606920 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:45.606747    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74668f54-rpscx_5389f834-f90e-4388-b2df-fbf5ea3ffe55/manager/0.log"
Jul 06 18:25:45.629923 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:45.629807    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74668f54-rpscx_5389f834-f90e-4388-b2df-fbf5ea3ffe55/copy-manifests/0.log"
Jul 06 18:25:45.736304 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:45.736200    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-ft2sq_338ff4a4-4c2b-42ef-9bd8-92e534da391d/postgres/0.log"
Jul 06 18:25:47.263710 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:47.263673    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-nhzmc_a96b501d-06cd-4cb1-adc7-6c11ea466ca6/openshift-lws-operator/0.log"
Jul 06 18:25:47.858198 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:47.858132    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-98dc6956-pmkj5_92359562-7966-400d-87ed-d7d35b885d40/kube-auth-proxy/0.log"
Jul 06 18:25:47.973088 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:47.973052    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-zr75s_f15e0340-5a61-44aa-b651-331f8a5c32fe/payload-pre-processing/0.log"
Jul 06 18:25:48.047112 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:48.047079    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-skm4k_89d6393c-dc4c-4cf1-bfaa-b389cc6fb439/payload-processing/0.log"
Jul 06 18:25:49.467760 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:49.467727    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-98dc6956-pmkj5_92359562-7966-400d-87ed-d7d35b885d40/kube-auth-proxy/0.log"
Jul 06 18:25:49.589152 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:49.589113    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-zr75s_f15e0340-5a61-44aa-b651-331f8a5c32fe/payload-pre-processing/0.log"
Jul 06 18:25:49.663461 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:49.663422    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-skm4k_89d6393c-dc4c-4cf1-bfaa-b389cc6fb439/payload-processing/0.log"
Jul 06 18:25:53.820433 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:53.820323    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4k5bv_ddafc9c5-84a7-432c-8907-41b4cbf2087a/kube-multus/0.log"
Jul 06 18:25:54.021253 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:54.021220    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfpln_a4d538c1-5ceb-4481-84e8-6d5d48883f7f/kube-multus-additional-cni-plugins/0.log"
Jul 06 18:25:54.044767 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:54.044724    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfpln_a4d538c1-5ceb-4481-84e8-6d5d48883f7f/egress-router-binary-copy/0.log"
Jul 06 18:25:54.071976 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:54.071870    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfpln_a4d538c1-5ceb-4481-84e8-6d5d48883f7f/cni-plugins/0.log"
Jul 06 18:25:54.097842 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:54.097815    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfpln_a4d538c1-5ceb-4481-84e8-6d5d48883f7f/bond-cni-plugin/0.log"
Jul 06 18:25:54.119350 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:54.119321    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfpln_a4d538c1-5ceb-4481-84e8-6d5d48883f7f/routeoverride-cni/0.log"
Jul 06 18:25:54.141723 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:54.141691    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfpln_a4d538c1-5ceb-4481-84e8-6d5d48883f7f/whereabouts-cni-bincopy/0.log"
Jul 06 18:25:54.164530 ip-10-0-133-236 kubenswrapper[2581]: I0706 18:25:54.164475    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mfpln_a4d538c1-5ceb-4481-84e8-6d5d48883f7f/whereabouts-cni/0.log"