Jul 09 12:00:16.707772 ip-10-0-141-94 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 09 12:00:16.707786 ip-10-0-141-94 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 09 12:00:16.707796 ip-10-0-141-94 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 09 12:00:16.708109 ip-10-0-141-94 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 09 12:00:26.751765 ip-10-0-141-94 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 09 12:00:26.751783 ip-10-0-141-94 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 47ef718e18d44bc0b5ef1851db6192bb --
Jul 09 12:02:07.184981 ip-10-0-141-94 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 12:02:07.672848 ip-10-0-141-94 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 12:02:07.672848 ip-10-0-141-94 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 12:02:07.672848 ip-10-0-141-94 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 12:02:07.672848 ip-10-0-141-94 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 12:02:07.672848 ip-10-0-141-94 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 12:02:07.673455 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.672907    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 12:02:07.679208 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679188    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 12:02:07.679208 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679204    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 12:02:07.679208 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679208    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679213    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679216    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679219    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679222    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679226    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679230    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679233    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679236    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679239    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679242    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679245    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679251    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679254    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679257    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679260    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679262    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679265    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679268    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 12:02:07.679311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679270    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679273    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679276    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679279    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679281    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679284    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679287    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679290    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679293    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679296    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679298    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679301    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679303    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679305    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679308    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679312    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679314    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679317    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679320    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679322    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 12:02:07.679770 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679326    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679328    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679330    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679333    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679336    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679338    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679341    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679343    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679346    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679348    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679351    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679355    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679358    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679360    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679363    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679365    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679368    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679370    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679373    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 12:02:07.680248 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679376    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679379    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679382    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679385    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679387    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679390    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679392    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679396    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679399    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679401    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679404    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679407    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679410    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679412    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679415    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679418    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679422    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679425    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679428    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679430    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 12:02:07.680815 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679433    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679436    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679438    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679440    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679443    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679445    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679447    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679450    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679452    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679454    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679457    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679459    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679462    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679464    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679467    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679469    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679471    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679474    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.679476    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 12:02:07.681342 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680180    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680187    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680190    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680193    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680195    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680198    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680200    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680203    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680205    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680208    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680210    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680213    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680217    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680219    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680222    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680225    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680227    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680229    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680232    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680234    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 12:02:07.681814 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680237    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680239    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680241    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680244    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680246    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680249    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680251    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680254    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680256    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680259    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680262    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680264    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680267    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680270    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680273    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680275    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680278    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680281    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680283    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680286    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 12:02:07.682305 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680288    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680290    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680294    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680298    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680301    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680304    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680307    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680309    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680312    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680314    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680317    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680319    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680322    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680324    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680327    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680329    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680332    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680334    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680336    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 12:02:07.682820 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680339    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680341    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680344    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680347    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680349    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680352    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680355    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680358    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680360    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680363    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680366    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680368    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680370    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680373    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680375    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680378    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680380    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680382    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680385    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680388    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 12:02:07.683281 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680390    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680392    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680394    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680397    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680399    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680401    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680405    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680411    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680415    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680418    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680421    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680424    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680426    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680429    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680431    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680434    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680437    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680442    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680445    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 12:02:07.683772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680447    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680814    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680819    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680822    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680825    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680827    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680830    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680833    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680835    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680839    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680843    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680846    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680849    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680852    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680855    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680857    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680860    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680862    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680864    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 12:02:07.684223 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680867    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680869    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680871    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680874    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680877    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680879    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680881    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680884    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680886    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680889    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680892    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680897    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680899    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680902    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680904    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680907    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680909    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680911    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680913    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680916    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 12:02:07.684683 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680918    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680921    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680923    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680926    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680928    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680932    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680939    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680942    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680945    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680947    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680950    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680953    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680955    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680958    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680960    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680962    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680965    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680967    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680969    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680972    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 12:02:07.685184 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680975    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680977    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680980    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680983    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680986    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680988    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680992    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680995    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.680998    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681000    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681003    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681005    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681008    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681010    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681012    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681015    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681018    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681020    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681022    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681025    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 12:02:07.685646 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681028    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681030    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681033    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681035    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681037    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681040    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681042    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681044    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681047    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681049    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681051    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681054    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681057    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681060    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681062    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681066    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681068    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681071    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681073    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681075    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 12:02:07.686212 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681078    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681150    2565 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681157    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681163    2565 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681167    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681171    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681174    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681178    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681183    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681186    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681189    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681192    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681196    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681199    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681201    2565 flags.go:64] FLAG: --cgroup-root=""
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681204    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681207    2565 flags.go:64] FLAG: --client-ca-file=""
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681210    2565 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681212    2565 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681215    2565 flags.go:64] FLAG: --cluster-domain=""
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681218    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681221    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681224    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681227    2565 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 12:02:07.686744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681232    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681234    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681237    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681242    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681244    2565 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681247    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681250    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681253    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681256    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681260    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681263    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681266    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681269    2565 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681271    2565 flags.go:64] FLAG: --enable-server="true"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681274    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681278    2565 flags.go:64] FLAG: --event-burst="100"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681281    2565 flags.go:64] FLAG: --event-qps="50"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681284    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681286    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681289    2565 flags.go:64] FLAG: --eviction-hard=""
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681293    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681296    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681298    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681301    2565 flags.go:64] FLAG: --eviction-soft=""
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681304    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 12:02:07.687331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681307    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681309    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681312    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681315    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681317    2565 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681320    2565 flags.go:64] FLAG: --feature-gates=""
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681323    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681327    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681330    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681333    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681337    2565 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681340    2565 flags.go:64] FLAG: --help="false"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681343    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-141-94.ec2.internal"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681346    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681349    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681352    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681355    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681358    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681361    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681364    2565 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681367    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681369    2565 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681372    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681375    2565 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 12:02:07.687969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681378    2565 flags.go:64] FLAG: --kube-reserved=""
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681380    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681383    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681386    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681388    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681392    2565 flags.go:64] FLAG: --lock-file=""
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681394    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681397    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681400    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681405    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681407    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681410    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681413    2565 flags.go:64] FLAG: --logging-format="text"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681416    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681419    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681422    2565 flags.go:64] FLAG: --manifest-url=""
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681425    2565 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681429    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681434    2565 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681437    2565 flags.go:64] FLAG: --max-pods="110"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681440    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681443    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681446    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681449    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681451    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 12:02:07.688589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681454    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681457    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681463    2565 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681466    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681469    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681472    2565 flags.go:64] FLAG: --pod-cidr=""
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681474    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681480    2565 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681482    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681485    2565 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681488    2565 flags.go:64] FLAG: --port="10250"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681491    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681494    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00e8f332662b4ac44"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681497    2565 flags.go:64] FLAG: --qos-reserved=""
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681500    2565 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681503    2565 flags.go:64] FLAG: --register-node="true"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681506    2565 flags.go:64] FLAG: --register-with-taints=""
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681509    2565 flags.go:64] FLAG: --registry-burst="10"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681511    2565 flags.go:64] FLAG: --registry-qps="5"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681514    2565 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681516    2565 flags.go:64] FLAG: --reserved-memory=""
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681520    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681523    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681527    2565 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681530    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 12:02:07.689430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681534    2565 flags.go:64] FLAG: --runonce="false"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681537    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681540    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681542    2565 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681545    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681548    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681551    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681553    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681556    2565 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681559    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681562    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681564    2565 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681567    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681570    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681572    2565 flags.go:64] FLAG: --system-cgroups=""
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681575    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681580    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681582    2565 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681585    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681589    2565 flags.go:64] FLAG: --tls-min-version=""
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681592    2565 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681595    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681597    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681600    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681603    2565 flags.go:64] FLAG: --v="2"
Jul 09 12:02:07.690205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681606    2565 flags.go:64] FLAG: --version="false"
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681610    2565 flags.go:64] FLAG: --vmodule=""
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681613    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.681616    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681744    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681749    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681752    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681759    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681763    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681766    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681768    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681771    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681774    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681777    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681779    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681782    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681785    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681787    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681790    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681792    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 12:02:07.690829 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681795    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681797    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681800    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681802    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681806    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681809    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681812    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681815    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681818    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681821    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681824    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681827    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681829    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681832    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681834    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681836    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681839    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681842    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681845    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 12:02:07.691303 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681851    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681855    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681858    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681860    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681862    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681865    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681867    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681870    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681872    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681875    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681877    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681880    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681882    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681884    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681887    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681889    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681892    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681895    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681897    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681899    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 12:02:07.691788 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681902    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681904    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681907    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681909    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681912    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681914    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681916    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681919    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681921    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681923    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681927    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681929    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681933    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681935    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681938    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681940    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681942    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681945    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681947    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681950    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 12:02:07.692269 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681952    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681955    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681957    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681960    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681962    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681964    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681967    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681970    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681972    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681974    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681977    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681979    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681982    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681984    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681987    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681989    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681992    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681994    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681996    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.681999    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 12:02:07.692775 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.682001    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.682003    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.682007    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.682009    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.682770    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.691691    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.691724    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.691741    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.691750    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691805    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691810    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691814    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691817    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691820    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691823    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 12:02:07.693243 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691825    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691828    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691830    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691833    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691835    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691838    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691840    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691842    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691845    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691847    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691850    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691853    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691855    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691857    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691860    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691863    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691866    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691868    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691871    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691873    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 12:02:07.693613 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691876    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691878    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691881    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691885    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691887    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691890    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691892    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691895    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691898    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691900    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691903    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691906    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691908    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691911    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691913    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691915    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691918    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691922    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691926    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 12:02:07.694102 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691929    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691931    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691933    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691936    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691939    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691941    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691945    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691949    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691952    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691954    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691957    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691960    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691963    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691965    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691968    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691971    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691973    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691976    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691979    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 12:02:07.694563 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691981    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691984    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691986    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691989    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691991    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691994    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691996    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.691998    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692000    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692003    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692005    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692008    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692010    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692012    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692015    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692018    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692021    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692024    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692026    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 12:02:07.695029 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692029    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692031    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692034    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692036    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692038    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692041    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692044    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692046    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692049    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692052    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692055    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692058    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692061    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692064    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692067    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 12:02:07.695522 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692070    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.692075    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692181    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692186    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692189    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692192    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692195    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692197    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692200    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692203    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692205    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692208    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692210    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692213    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692217    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 12:02:07.695996 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692221    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692224    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692226    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692230    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692234    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692237    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692239    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692242    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692244    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692247    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692250    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692253    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692255    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692258    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692260    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692263    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692265    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692267    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692270    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692272    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 12:02:07.696368 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692275    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692278    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692280    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692283    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692285    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692288    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692290    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692294    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692298    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692300    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692303    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692306    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692309    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692312    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692314    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692316    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692319    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692322    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692324    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 12:02:07.696869 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692326    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692329    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692331    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692334    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692338    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692341    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692343    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692346    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692349    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692352    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692355    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692357    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692359    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692362    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692364    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692366    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692369    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692371    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692374    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 12:02:07.697309 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692376    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692379    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692381    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692384    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692386    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692389    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692391    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692394    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692397    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692399    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692402    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692404    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692406    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692409    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692411    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692414    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692416    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692418    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692421    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692423    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 12:02:07.697785 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692425    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 12:02:07.698298 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692428    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 12:02:07.698298 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692430    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 12:02:07.698298 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692433    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 12:02:07.698298 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692435    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 12:02:07.698298 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692437    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 12:02:07.698298 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692440    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 12:02:07.698298 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:07.692442    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 12:02:07.698298 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.692447    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 12:02:07.698298 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.692557    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 12:02:07.698298 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.698065    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 12:02:07.700354 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.700342    2565 server.go:1019] "Starting client certificate rotation"
Jul 09 12:02:07.700461 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.700444    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 12:02:07.700491 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.700477    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 12:02:07.704709 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.704683    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 12:02:07.705966 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.705947    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 12:02:07.724499 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.724479    2565 log.go:25] "Validated CRI v1 runtime API"
Jul 09 12:02:07.726871 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.726856    2565 log.go:25] "Validated CRI v1 image API"
Jul 09 12:02:07.727654 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.727636    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 12:02:07.730232 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.730206    2565 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 830a46d7-f4a2-49e2-9e6f-adbf7cff4a68:/dev/nvme0n1p4 eb9d9a88-66a1-41bf-9eb9-16be6a66f1c7:/dev/nvme0n1p3]
Jul 09 12:02:07.730304 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.730233    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 12:02:07.732383 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.732358    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 12:02:07.736912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.736803    2565 manager.go:216] Machine: {Timestamp:2026-07-09 12:02:07.735374489 +0000 UTC m=+0.412903538 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098112 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec244fedd909514c7fbd66200f03ecc6 SystemUUID:ec244fed-d909-514c-7fbd-66200f03ecc6 BootID:47ef718e-18d4-4bc0-b5ef-1851db6192bb Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:43:16:e3:47:73 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:43:16:e3:47:73 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:86:59:c0:05:89:c8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 12:02:07.736912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.736905    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 12:02:07.737038 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.737011    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 12:02:07.741138 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.741111    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 12:02:07.741268 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.741141    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-94.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 12:02:07.741949 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.741939    2565 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 12:02:07.741980 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.741951    2565 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 12:02:07.741980 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.741959    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 12:02:07.743165 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.743154    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 12:02:07.747581 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.747571    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 12:02:07.751398 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.751387    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 12:02:07.751509 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.751500    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 12:02:07.751545 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.751530    2565 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 12:02:07.751545 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.751541    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 12:02:07.751600 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.751555    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 12:02:07.751600 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.751563    2565 kubelet.go:398] "Adding apiserver pod source"
Jul 09 12:02:07.751600 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.751571    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 12:02:07.751913 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.751886    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 12:02:07.751913 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.751912    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 12:02:07.754714 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.754684    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 09 12:02:07.756779 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.756762    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 12:02:07.756866 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.756801    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 12:02:07.758172 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.758158    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 12:02:07.758264 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.758180    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 12:02:07.758264 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.758189    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 12:02:07.758264 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.758198    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 12:02:07.758264 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.758211    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 12:02:07.758264 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.758219    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 12:02:07.758264 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.758232    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 12:02:07.758264 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.758240    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 12:02:07.758264 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.758249    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 12:02:07.758264 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.758257    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 12:02:07.758541 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.758276    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 12:02:07.758541 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.758285    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 12:02:07.759778 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.759763    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x4hk8"
Jul 09 12:02:07.759983 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.759969    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 12:02:07.760037 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.759990    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 12:02:07.760386 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:07.760365    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 12:02:07.760474 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:07.760423    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-94.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 12:02:07.762409 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.762391    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-94.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 12:02:07.763244 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.763229    2565 server.go:1268] "Started kubelet"
Jul 09 12:02:07.763348 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.763320    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 12:02:07.763394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.763313    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 12:02:07.763431 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.763410    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 12:02:07.764079 ip-10-0-141-94 systemd[1]: Started Kubernetes Kubelet.
Jul 09 12:02:07.764872 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.764790    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 12:02:07.767064 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.767048    2565 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 12:02:07.767557 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.767543    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x4hk8"
Jul 09 12:02:07.769078 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:07.769056    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 12:02:07.770800 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:07.769767    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-94.ec2.internal.18c09d693d3c41e3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-94.ec2.internal,UID:ip-10-0-141-94.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-94.ec2.internal,},FirstTimestamp:2026-07-09 12:02:07.763202531 +0000 UTC m=+0.440731581,LastTimestamp:2026-07-09 12:02:07.763202531 +0000 UTC m=+0.440731581,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-94.ec2.internal,}"
Jul 09 12:02:07.773574 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.773557    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 12:02:07.773650 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.773615    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 12:02:07.773997 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:07.773978    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-94.ec2.internal\" not found"
Jul 09 12:02:07.774300 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.774279    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 12:02:07.774455 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.774308    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 12:02:07.774579 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.774551    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 12:02:07.774792 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.774748    2565 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 12:02:07.774792 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.774763    2565 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 12:02:07.774941 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.774917    2565 factory.go:55] Registering systemd factory
Jul 09 12:02:07.774941 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.774931    2565 factory.go:223] Registration of the systemd container factory successfully
Jul 09 12:02:07.775174 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.775108    2565 factory.go:153] Registering CRI-O factory
Jul 09 12:02:07.775174 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.775122    2565 factory.go:223] Registration of the crio container factory successfully
Jul 09 12:02:07.775286 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.775183    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 12:02:07.775286 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.775212    2565 factory.go:103] Registering Raw factory
Jul 09 12:02:07.775286 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.775228    2565 manager.go:1196] Started watching for new ooms in manager
Jul 09 12:02:07.775872 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.775854    2565 manager.go:349] Starting recovery of all containers
Jul 09 12:02:07.786456 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.786426    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 12:02:07.789394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.789379    2565 manager.go:354] Recovery completed
Jul 09 12:02:07.791634 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:07.791615    2565 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-94.ec2.internal\" not found" node="ip-10-0-141-94.ec2.internal"
Jul 09 12:02:07.793789 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.793775    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:02:07.797531 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.797514    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:02:07.797603 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.797549    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:02:07.797603 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.797559    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:02:07.798147 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.798131    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 12:02:07.798147 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.798143    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 12:02:07.798246 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.798159    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 12:02:07.798246 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.798227    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:02:07.798979 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.798964    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:02:07.799035 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.798995    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:02:07.799035 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.799011    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:02:07.799449 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.799437    2565 policy_none.go:49] "None policy: Start"
Jul 09 12:02:07.799481 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.799454    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 12:02:07.799481 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.799465    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 12:02:07.800736 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.800726    2565 policy_none.go:47] "Start"
Jul 09 12:02:07.841466 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.841304    2565 manager.go:337] "Starting Device Plugin manager"
Jul 09 12:02:07.851540 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:07.841488    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 12:02:07.851540 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.841502    2565 server.go:87] "Starting device plugin registration server"
Jul 09 12:02:07.851540 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.841789    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 12:02:07.851540 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.841803    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 12:02:07.851540 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.841915    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 12:02:07.851540 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.842016    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 12:02:07.851540 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.842026    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 12:02:07.851540 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:07.842503    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 12:02:07.851540 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:07.842564    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-94.ec2.internal\" not found"
Jul 09 12:02:07.885939 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.885913    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 12:02:07.887264 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.887247    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 12:02:07.887353 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.887279    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 12:02:07.887353 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.887297    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 12:02:07.887353 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:07.887330    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 12:02:07.889463 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.889443    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 12:02:07.942441 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.942389    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:02:07.943184 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.943169    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:02:07.943266 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.943203    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:02:07.943266 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.943217    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:02:07.943266 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.943245    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-94.ec2.internal"
Jul 09 12:02:07.954040 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.954019    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-94.ec2.internal"
Jul 09 12:02:07.954040 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:07.954040    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-94.ec2.internal\": node \"ip-10-0-141-94.ec2.internal\" not found"
Jul 09 12:02:07.987922 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.987891    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-94.ec2.internal"]
Jul 09 12:02:07.987998 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.987953    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:02:07.988618 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.988605    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:02:07.988681 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.988641    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:02:07.988681 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.988652    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:02:07.989399 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:07.989383    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-94.ec2.internal\" not found"
Jul 09 12:02:07.990198 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.990186    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:02:07.990300 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.990289    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:07.990338 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.990315    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:02:07.990878 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.990863    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:02:07.990878 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.990872    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:02:07.991035 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.990887    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:02:07.991035 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.990896    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:02:07.991035 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.990896    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:02:07.991035 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.990910    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:02:07.992409 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.992393    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:07.992459 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.992431    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 12:02:07.993066 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.993049    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 12:02:07.993147 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.993082    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 12:02:07.993147 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:07.993097    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeHasSufficientPID"
Jul 09 12:02:08.016382 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:08.016358    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-94.ec2.internal\" not found" node="ip-10-0-141-94.ec2.internal"
Jul 09 12:02:08.020619 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:08.020604    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-94.ec2.internal\" not found" node="ip-10-0-141-94.ec2.internal"
Jul 09 12:02:08.076333 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.076309    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e001a5302fdde64d4d52a11b0c122200-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal\" (UID: \"e001a5302fdde64d4d52a11b0c122200\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:08.076420 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.076335    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e001a5302fdde64d4d52a11b0c122200-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal\" (UID: \"e001a5302fdde64d4d52a11b0c122200\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:08.076420 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.076352    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ed9a03f52aaae8a3339d4a55bea80f61-config\") pod \"kube-apiserver-proxy-ip-10-0-141-94.ec2.internal\" (UID: \"ed9a03f52aaae8a3339d4a55bea80f61\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:08.090410 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:08.090391    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-94.ec2.internal\" not found"
Jul 09 12:02:08.177383 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.177355    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e001a5302fdde64d4d52a11b0c122200-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal\" (UID: \"e001a5302fdde64d4d52a11b0c122200\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:08.177514 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.177387    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e001a5302fdde64d4d52a11b0c122200-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal\" (UID: \"e001a5302fdde64d4d52a11b0c122200\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:08.177514 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.177411    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ed9a03f52aaae8a3339d4a55bea80f61-config\") pod \"kube-apiserver-proxy-ip-10-0-141-94.ec2.internal\" (UID: \"ed9a03f52aaae8a3339d4a55bea80f61\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:08.177514 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.177445    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ed9a03f52aaae8a3339d4a55bea80f61-config\") pod \"kube-apiserver-proxy-ip-10-0-141-94.ec2.internal\" (UID: \"ed9a03f52aaae8a3339d4a55bea80f61\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:08.177514 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.177448    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e001a5302fdde64d4d52a11b0c122200-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal\" (UID: \"e001a5302fdde64d4d52a11b0c122200\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:08.177514 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.177454    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e001a5302fdde64d4d52a11b0c122200-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal\" (UID: \"e001a5302fdde64d4d52a11b0c122200\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:08.191498 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:08.191479    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-94.ec2.internal\" not found"
Jul 09 12:02:08.292277 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:08.292211    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-94.ec2.internal\" not found"
Jul 09 12:02:08.319376 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.319355    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:08.323895 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.323727    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:08.392513 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:08.392490    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-94.ec2.internal\" not found"
Jul 09 12:02:08.492980 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:08.492955    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-94.ec2.internal\" not found"
Jul 09 12:02:08.593480 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:08.593414    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-94.ec2.internal\" not found"
Jul 09 12:02:08.693919 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:08.693893    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-94.ec2.internal\" not found"
Jul 09 12:02:08.699059 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.699042    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 12:02:08.699213 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.699195    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 12:02:08.699213 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.699201    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 12:02:08.772865 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.772833    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 11:57:08 +0000 UTC" deadline="2028-02-15 12:19:42.503883137 +0000 UTC"
Jul 09 12:02:08.772865 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.772861    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14064h17m33.731026132s"
Jul 09 12:02:08.793974 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:08.793939    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-94.ec2.internal\" not found"
Jul 09 12:02:08.866944 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.866873    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 12:02:08.894134 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:08.894110    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-94.ec2.internal\" not found"
Jul 09 12:02:08.994814 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:08.994779    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-94.ec2.internal\" not found"
Jul 09 12:02:08.995396 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:08.995381    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 12:02:09.057214 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:09.057183    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode001a5302fdde64d4d52a11b0c122200.slice/crio-c0141af52b873c664b42bbfad9367c80cc2b0facabe940530c8892a08980bfb8 WatchSource:0}: Error finding container c0141af52b873c664b42bbfad9367c80cc2b0facabe940530c8892a08980bfb8: Status 404 returned error can't find the container with id c0141af52b873c664b42bbfad9367c80cc2b0facabe940530c8892a08980bfb8
Jul 09 12:02:09.057740 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:09.057725    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded9a03f52aaae8a3339d4a55bea80f61.slice/crio-95e5c543c8a0e25591e6ec43b628a78230f24c75cc86fac6551881b9fc439647 WatchSource:0}: Error finding container 95e5c543c8a0e25591e6ec43b628a78230f24c75cc86fac6551881b9fc439647: Status 404 returned error can't find the container with id 95e5c543c8a0e25591e6ec43b628a78230f24c75cc86fac6551881b9fc439647
Jul 09 12:02:09.062486 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.062472    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 12:02:09.075055 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.075034    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:09.085900 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.085883    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 12:02:09.086889 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.086877    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-94.ec2.internal"
Jul 09 12:02:09.095596 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.095581    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 12:02:09.753363 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.753146    2565 apiserver.go:52] "Watching apiserver"
Jul 09 12:02:09.763358 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.762408    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 12:02:09.763358 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.762932    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-qf2v9","openshift-cluster-node-tuning-operator/tuned-pcvzf","openshift-dns/node-resolver-zjcb9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal","openshift-multus/multus-mn4b9","openshift-multus/network-metrics-daemon-rltxg","openshift-network-diagnostics/network-check-target-p9wwl","openshift-network-operator/iptables-alerter-rlbpb","openshift-ovn-kubernetes/ovnkube-node-l2hg7","kube-system/global-pull-secret-syncer-p456j","kube-system/konnectivity-agent-q4n45","kube-system/kube-apiserver-proxy-ip-10-0-141-94.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq","openshift-image-registry/node-ca-r5k4m"]
Jul 09 12:02:09.765314 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.764887    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:09.765314 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:09.764975    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:09.766547 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.766524    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.768438 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.768189    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zjcb9"
Jul 09 12:02:09.769867 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.769846    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r5k4m"
Jul 09 12:02:09.770084 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.770061    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cnscz\""
Jul 09 12:02:09.770431 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.770412    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 12:02:09.770618 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.770604    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 12:02:09.770721 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.770688    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lrwnt\""
Jul 09 12:02:09.770818 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.770806    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 12:02:09.770920 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.770903    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 12:02:09.771100 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.771073    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 12:02:09.771682 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.771662    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.772793 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.772761    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 12:02:09.773027 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.772999    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cmkmw\""
Jul 09 12:02:09.773272 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.773253    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 12:02:09.773522 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.773504    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 12:02:09.773968 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.773949    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:09.774048 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:09.774015    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:09.774108 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.774063    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 12:02:09.774514 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.774485    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.775909 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.775879    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gv5fl\""
Jul 09 12:02:09.776058 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.776039    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 12:02:09.776178 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.776147    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 12:02:09.776823 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.776249    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 12:02:09.776823 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.776422    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 12:02:09.776823 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.776524    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:09.776823 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:09.776588    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:09.777209 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.777191    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 12:02:09.777334 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.777319    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tlzd8\""
Jul 09 12:02:09.777430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.777325    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 12:02:09.779211 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.778386    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.779211 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.778410    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 12:02:09.779211 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.778609    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 12:02:09.779211 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.778971    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 12:02:09.779211 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.779011    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785458    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2-host\") pod \"node-ca-r5k4m\" (UID: \"8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2\") " pod="openshift-image-registry/node-ca-r5k4m"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785507    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-os-release\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785539    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdwpw\" (UniqueName: \"kubernetes.io/projected/b8802ef7-f67c-4aee-81cd-858b426464d9-kube-api-access-cdwpw\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785578    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-systemd-units\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785603    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-node-log\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785634    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785663    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-cni-bin\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785693    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785729    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-dbus\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785754    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-cnibin\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785808    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-run-netns\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785840    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-sys-fs\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785870    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjwxs\" (UniqueName: \"kubernetes.io/projected/7d3e98cc-7393-442a-9b2c-02443b8c3b64-kube-api-access-kjwxs\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785917    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjssj\" (UniqueName: \"kubernetes.io/projected/760d11e4-c839-41a2-8e35-b4d2566ef960-kube-api-access-mjssj\") pod \"node-resolver-zjcb9\" (UID: \"760d11e4-c839-41a2-8e35-b4d2566ef960\") " pod="openshift-dns/node-resolver-zjcb9"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785942    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-etc-kubernetes\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.785973    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/760d11e4-c839-41a2-8e35-b4d2566ef960-hosts-file\") pod \"node-resolver-zjcb9\" (UID: \"760d11e4-c839-41a2-8e35-b4d2566ef960\") " pod="openshift-dns/node-resolver-zjcb9"
Jul 09 12:02:09.787263 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786001    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/760d11e4-c839-41a2-8e35-b4d2566ef960-tmp-dir\") pod \"node-resolver-zjcb9\" (UID: \"760d11e4-c839-41a2-8e35-b4d2566ef960\") " pod="openshift-dns/node-resolver-zjcb9"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786033    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-kubelet-config\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786064    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-run-systemd\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786096    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x28d\" (UniqueName: \"kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d\") pod \"network-check-target-p9wwl\" (UID: \"4afb2515-0939-4e93-9d3d-7f5662d0856e\") " pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786123    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-var-lib-cni-bin\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786173    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6b958117-535b-49f3-b74c-4d809655d7d1-env-overrides\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786205    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs\") pod \"network-metrics-daemon-rltxg\" (UID: \"4ac6e98e-e488-4741-aa67-2a6a130291d8\") " pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786234    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-run-ovn-kubernetes\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786268    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786307    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8802ef7-f67c-4aee-81cd-858b426464d9-multus-daemon-config\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786345    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-multus-cni-dir\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786384    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-multus-socket-dir-parent\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786424    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-run-netns\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786456    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-var-lib-kubelet\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786495    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-registration-dir\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786534    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2-serviceca\") pod \"node-ca-r5k4m\" (UID: \"8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2\") " pod="openshift-image-registry/node-ca-r5k4m"
Jul 09 12:02:09.788056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786574    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6b958117-535b-49f3-b74c-4d809655d7d1-ovn-node-metrics-cert\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786610    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngfs5\" (UniqueName: \"kubernetes.io/projected/6b958117-535b-49f3-b74c-4d809655d7d1-kube-api-access-ngfs5\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786639    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc6c2\" (UniqueName: \"kubernetes.io/projected/4ac6e98e-e488-4741-aa67-2a6a130291d8-kube-api-access-xc6c2\") pod \"network-metrics-daemon-rltxg\" (UID: \"4ac6e98e-e488-4741-aa67-2a6a130291d8\") " pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786957    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-kubelet\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.786991    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-etc-openvswitch\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787023    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-run-ovn\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787127    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-run-multus-certs\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787228    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-multus-conf-dir\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787282    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-var-lib-openvswitch\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787341    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-cni-netd\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787407    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-socket-dir\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787440    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-device-dir\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787465    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-etc-selinux\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787508    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-log-socket\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787540    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-slash\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787569    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-run-openvswitch\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.788820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787618    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6b958117-535b-49f3-b74c-4d809655d7d1-ovnkube-script-lib\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.789413 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787643    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6kt9\" (UniqueName: \"kubernetes.io/projected/8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2-kube-api-access-s6kt9\") pod \"node-ca-r5k4m\" (UID: \"8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2\") " pod="openshift-image-registry/node-ca-r5k4m"
Jul 09 12:02:09.789413 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787662    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-system-cni-dir\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.789413 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787687    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6b958117-535b-49f3-b74c-4d809655d7d1-ovnkube-config\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.789413 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787731    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8802ef7-f67c-4aee-81cd-858b426464d9-cni-binary-copy\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.789413 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787782    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-run-k8s-cni-cncf-io\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.789413 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787861    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-var-lib-cni-multus\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.789413 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787940    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 12:02:09.789413 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.787917    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-hostroot\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.789413 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.788102    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qkmsd\""
Jul 09 12:02:09.789413 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.788160    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 12:02:09.789413 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.788340    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q4n45"
Jul 09 12:02:09.790652 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.789907    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.791838 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.791816    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rlbpb"
Jul 09 12:02:09.792682 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.792660    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8gdkq\""
Jul 09 12:02:09.792682 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.792677    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 12:02:09.792888 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.792693    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 12:02:09.793015 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.792995    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 12:02:09.793126 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.793051    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 12:02:09.794351 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.794330    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jcxq9\""
Jul 09 12:02:09.795322 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.795286    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 12:02:09.795724 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.795535    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 12:02:09.795724 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.795559    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vb4v5\""
Jul 09 12:02:09.795724 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.795611    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 12:02:09.796796 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.796741    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 12:02:09.814267 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.814239    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4fjvh"
Jul 09 12:02:09.820561 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.820540    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4fjvh"
Jul 09 12:02:09.876073 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.876049    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 12:02:09.888297 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888261    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b994ce8a-e5c9-4828-b792-c9322dd75c26-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.888398 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888305    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:09.888398 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888333    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-cni-bin\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.888398 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888362    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.888398 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888389    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-dbus\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:09.888665 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888415    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-cnibin\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.888665 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:09.888418    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 12:02:09.888665 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888444    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-cni-bin\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.888665 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888446    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.888665 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888455    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-run-netns\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.888665 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888493    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-run-netns\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.888665 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:09.888529    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret podName:e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b nodeName:}" failed. No retries permitted until 2026-07-09 12:02:10.38849014 +0000 UTC m=+3.066019181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret") pod "global-pull-secret-syncer-p456j" (UID: "e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b") : object "kube-system"/"original-pull-secret" not registered
Jul 09 12:02:09.888665 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888555    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-sys-fs\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.888665 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888563    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-cnibin\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.888665 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888591    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjwxs\" (UniqueName: \"kubernetes.io/projected/7d3e98cc-7393-442a-9b2c-02443b8c3b64-kube-api-access-kjwxs\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.888665 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888614    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-sys-fs\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.888665 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888619    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjssj\" (UniqueName: \"kubernetes.io/projected/760d11e4-c839-41a2-8e35-b4d2566ef960-kube-api-access-mjssj\") pod \"node-resolver-zjcb9\" (UID: \"760d11e4-c839-41a2-8e35-b4d2566ef960\") " pod="openshift-dns/node-resolver-zjcb9"
Jul 09 12:02:09.888665 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888627    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-dbus\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:09.888665 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888662    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-kubernetes\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888694    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b994ce8a-e5c9-4828-b792-c9322dd75c26-cnibin\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888728    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-etc-kubernetes\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888747    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/760d11e4-c839-41a2-8e35-b4d2566ef960-hosts-file\") pod \"node-resolver-zjcb9\" (UID: \"760d11e4-c839-41a2-8e35-b4d2566ef960\") " pod="openshift-dns/node-resolver-zjcb9"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888790    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-etc-kubernetes\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888803    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/760d11e4-c839-41a2-8e35-b4d2566ef960-tmp-dir\") pod \"node-resolver-zjcb9\" (UID: \"760d11e4-c839-41a2-8e35-b4d2566ef960\") " pod="openshift-dns/node-resolver-zjcb9"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888841    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-var-lib-kubelet\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888853    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/760d11e4-c839-41a2-8e35-b4d2566ef960-hosts-file\") pod \"node-resolver-zjcb9\" (UID: \"760d11e4-c839-41a2-8e35-b4d2566ef960\") " pod="openshift-dns/node-resolver-zjcb9"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888872    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-kubelet-config\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888897    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-run-systemd\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888928    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-sysctl-conf\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888936    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-kubelet-config\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888953    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-lib-modules\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888981    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-run-systemd\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.888984    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x28d\" (UniqueName: \"kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d\") pod \"network-check-target-p9wwl\" (UID: \"4afb2515-0939-4e93-9d3d-7f5662d0856e\") " pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889018    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-var-lib-cni-bin\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889048    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/760d11e4-c839-41a2-8e35-b4d2566ef960-tmp-dir\") pod \"node-resolver-zjcb9\" (UID: \"760d11e4-c839-41a2-8e35-b4d2566ef960\") " pod="openshift-dns/node-resolver-zjcb9"
Jul 09 12:02:09.889469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889057    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-var-lib-cni-bin\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889090    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6b958117-535b-49f3-b74c-4d809655d7d1-env-overrides\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889119    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8e18877-0f68-4aad-86dc-5fba3a368d76-host-slash\") pod \"iptables-alerter-rlbpb\" (UID: \"f8e18877-0f68-4aad-86dc-5fba3a368d76\") " pod="openshift-network-operator/iptables-alerter-rlbpb"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889146    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs\") pod \"network-metrics-daemon-rltxg\" (UID: \"4ac6e98e-e488-4741-aa67-2a6a130291d8\") " pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889185    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-run-ovn-kubernetes\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889219    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889221    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-run-ovn-kubernetes\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889267    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-sysctl-d\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:09.889303    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889361    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889418    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-tuned\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889458    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fnrs\" (UniqueName: \"kubernetes.io/projected/f8e18877-0f68-4aad-86dc-5fba3a368d76-kube-api-access-9fnrs\") pod \"iptables-alerter-rlbpb\" (UID: \"f8e18877-0f68-4aad-86dc-5fba3a368d76\") " pod="openshift-network-operator/iptables-alerter-rlbpb"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889505    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6b958117-535b-49f3-b74c-4d809655d7d1-env-overrides\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:09.889514    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs podName:4ac6e98e-e488-4741-aa67-2a6a130291d8 nodeName:}" failed. No retries permitted until 2026-07-09 12:02:10.389491738 +0000 UTC m=+3.067020780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs") pod "network-metrics-daemon-rltxg" (UID: "4ac6e98e-e488-4741-aa67-2a6a130291d8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889560    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8802ef7-f67c-4aee-81cd-858b426464d9-multus-daemon-config\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889589    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-multus-cni-dir\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889620    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-multus-socket-dir-parent\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.890394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889642    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-run-netns\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889652    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-multus-cni-dir\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889668    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-var-lib-kubelet\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889699    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-registration-dir\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889739    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-multus-socket-dir-parent\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889750    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-run-netns\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889762    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-tmp\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889803    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-var-lib-kubelet\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889812    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d007f736-c0e5-4cf1-a25c-2dd14dbacb72-konnectivity-ca\") pod \"konnectivity-agent-q4n45\" (UID: \"d007f736-c0e5-4cf1-a25c-2dd14dbacb72\") " pod="kube-system/konnectivity-agent-q4n45"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889821    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-registration-dir\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889832    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2-serviceca\") pod \"node-ca-r5k4m\" (UID: \"8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2\") " pod="openshift-image-registry/node-ca-r5k4m"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889860    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6b958117-535b-49f3-b74c-4d809655d7d1-ovn-node-metrics-cert\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889878    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngfs5\" (UniqueName: \"kubernetes.io/projected/6b958117-535b-49f3-b74c-4d809655d7d1-kube-api-access-ngfs5\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889897    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-sysconfig\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889916    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-systemd\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889944    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b994ce8a-e5c9-4828-b792-c9322dd75c26-system-cni-dir\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889975    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b994ce8a-e5c9-4828-b792-c9322dd75c26-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.891303 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.889997    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc6c2\" (UniqueName: \"kubernetes.io/projected/4ac6e98e-e488-4741-aa67-2a6a130291d8-kube-api-access-xc6c2\") pod \"network-metrics-daemon-rltxg\" (UID: \"4ac6e98e-e488-4741-aa67-2a6a130291d8\") " pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890013    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-kubelet\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890041    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-etc-openvswitch\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890063    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-run-ovn\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890080    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzdlf\" (UniqueName: \"kubernetes.io/projected/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-kube-api-access-gzdlf\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890097    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-run-multus-certs\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890118    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-multus-conf-dir\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890143    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-var-lib-openvswitch\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890162    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-cni-netd\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890167    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8802ef7-f67c-4aee-81cd-858b426464d9-multus-daemon-config\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890184    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-socket-dir\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890197    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-kubelet\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890219    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-device-dir\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890247    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-etc-selinux\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890273    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-socket-dir\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890270    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890289    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-log-socket\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890328    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-etc-openvswitch\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890351    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-modprobe-d\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890374    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-run\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890401    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-sys\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890419    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-var-lib-openvswitch\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890425    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-slash\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890452    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-run-openvswitch\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890465    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-cni-netd\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890373    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-run-multus-certs\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890484    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-etc-selinux\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890477    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6b958117-535b-49f3-b74c-4d809655d7d1-ovnkube-script-lib\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890545    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-run-ovn\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890548    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-host-slash\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890532    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b994ce8a-e5c9-4828-b792-c9322dd75c26-cni-binary-copy\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890318    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7d3e98cc-7393-442a-9b2c-02443b8c3b64-device-dir\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890546    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-log-socket\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890595    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-multus-conf-dir\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890603    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-run-openvswitch\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.892912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890618    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8e18877-0f68-4aad-86dc-5fba3a368d76-iptables-alerter-script\") pod \"iptables-alerter-rlbpb\" (UID: \"f8e18877-0f68-4aad-86dc-5fba3a368d76\") " pod="openshift-network-operator/iptables-alerter-rlbpb"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890666    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6kt9\" (UniqueName: \"kubernetes.io/projected/8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2-kube-api-access-s6kt9\") pod \"node-ca-r5k4m\" (UID: \"8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2\") " pod="openshift-image-registry/node-ca-r5k4m"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890710    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-system-cni-dir\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890757    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6b958117-535b-49f3-b74c-4d809655d7d1-ovnkube-config\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890818    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8802ef7-f67c-4aee-81cd-858b426464d9-cni-binary-copy\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890844    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-run-k8s-cni-cncf-io\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890894    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-system-cni-dir\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890963    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-var-lib-cni-multus\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.890987    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-hostroot\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891016    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5z9f\" (UniqueName: \"kubernetes.io/projected/b994ce8a-e5c9-4828-b792-c9322dd75c26-kube-api-access-r5z9f\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891073    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d007f736-c0e5-4cf1-a25c-2dd14dbacb72-agent-certs\") pod \"konnectivity-agent-q4n45\" (UID: \"d007f736-c0e5-4cf1-a25c-2dd14dbacb72\") " pod="kube-system/konnectivity-agent-q4n45"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891101    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2-host\") pod \"node-ca-r5k4m\" (UID: \"8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2\") " pod="openshift-image-registry/node-ca-r5k4m"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891110    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2-serviceca\") pod \"node-ca-r5k4m\" (UID: \"8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2\") " pod="openshift-image-registry/node-ca-r5k4m"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891130    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-os-release\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891155    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-host\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891147    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-var-lib-cni-multus\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891179    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-hostroot\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891184    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b994ce8a-e5c9-4828-b792-c9322dd75c26-os-release\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.893637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891254    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2-host\") pod \"node-ca-r5k4m\" (UID: \"8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2\") " pod="openshift-image-registry/node-ca-r5k4m"
Jul 09 12:02:09.894549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891268    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdwpw\" (UniqueName: \"kubernetes.io/projected/b8802ef7-f67c-4aee-81cd-858b426464d9-kube-api-access-cdwpw\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.894549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891285    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6b958117-535b-49f3-b74c-4d809655d7d1-ovnkube-script-lib\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.894549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891297    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-host-run-k8s-cni-cncf-io\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.894549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891297    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-systemd-units\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.894549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891338    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-systemd-units\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.894549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891379    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-node-log\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.894549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891407    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b994ce8a-e5c9-4828-b792-c9322dd75c26-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.894549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891467    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8802ef7-f67c-4aee-81cd-858b426464d9-cni-binary-copy\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.894549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891512    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8802ef7-f67c-4aee-81cd-858b426464d9-os-release\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.894549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891559    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6b958117-535b-49f3-b74c-4d809655d7d1-node-log\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.894549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.891820    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6b958117-535b-49f3-b74c-4d809655d7d1-ovnkube-config\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.894549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.892612    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-94.ec2.internal" event={"ID":"ed9a03f52aaae8a3339d4a55bea80f61","Type":"ContainerStarted","Data":"95e5c543c8a0e25591e6ec43b628a78230f24c75cc86fac6551881b9fc439647"}
Jul 09 12:02:09.894549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.894265    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal" event={"ID":"e001a5302fdde64d4d52a11b0c122200","Type":"ContainerStarted","Data":"c0141af52b873c664b42bbfad9367c80cc2b0facabe940530c8892a08980bfb8"}
Jul 09 12:02:09.894549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.894482    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6b958117-535b-49f3-b74c-4d809655d7d1-ovn-node-metrics-cert\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.897446 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:09.897422    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 12:02:09.897534 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:09.897451    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 12:02:09.897534 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:09.897467    2565 projected.go:196] Error preparing data for projected volume kube-api-access-8x28d for pod openshift-network-diagnostics/network-check-target-p9wwl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:02:09.897604 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:09.897545    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d podName:4afb2515-0939-4e93-9d3d-7f5662d0856e nodeName:}" failed. No retries permitted until 2026-07-09 12:02:10.39752873 +0000 UTC m=+3.075057791 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8x28d" (UniqueName: "kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d") pod "network-check-target-p9wwl" (UID: "4afb2515-0939-4e93-9d3d-7f5662d0856e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:02:09.898046 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.898022    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjssj\" (UniqueName: \"kubernetes.io/projected/760d11e4-c839-41a2-8e35-b4d2566ef960-kube-api-access-mjssj\") pod \"node-resolver-zjcb9\" (UID: \"760d11e4-c839-41a2-8e35-b4d2566ef960\") " pod="openshift-dns/node-resolver-zjcb9"
Jul 09 12:02:09.898389 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.898367    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjwxs\" (UniqueName: \"kubernetes.io/projected/7d3e98cc-7393-442a-9b2c-02443b8c3b64-kube-api-access-kjwxs\") pod \"aws-ebs-csi-driver-node-mxckq\" (UID: \"7d3e98cc-7393-442a-9b2c-02443b8c3b64\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:09.899582 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.899372    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngfs5\" (UniqueName: \"kubernetes.io/projected/6b958117-535b-49f3-b74c-4d809655d7d1-kube-api-access-ngfs5\") pod \"ovnkube-node-l2hg7\" (UID: \"6b958117-535b-49f3-b74c-4d809655d7d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:09.900639 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.900612    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6kt9\" (UniqueName: \"kubernetes.io/projected/8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2-kube-api-access-s6kt9\") pod \"node-ca-r5k4m\" (UID: \"8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2\") " pod="openshift-image-registry/node-ca-r5k4m"
Jul 09 12:02:09.900860 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.900835    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc6c2\" (UniqueName: \"kubernetes.io/projected/4ac6e98e-e488-4741-aa67-2a6a130291d8-kube-api-access-xc6c2\") pod \"network-metrics-daemon-rltxg\" (UID: \"4ac6e98e-e488-4741-aa67-2a6a130291d8\") " pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:09.902079 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.902059    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdwpw\" (UniqueName: \"kubernetes.io/projected/b8802ef7-f67c-4aee-81cd-858b426464d9-kube-api-access-cdwpw\") pod \"multus-mn4b9\" (UID: \"b8802ef7-f67c-4aee-81cd-858b426464d9\") " pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:09.948014 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.947982    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 12:02:09.992044 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992011    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b994ce8a-e5c9-4828-b792-c9322dd75c26-cni-binary-copy\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.992190 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992055    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8e18877-0f68-4aad-86dc-5fba3a368d76-iptables-alerter-script\") pod \"iptables-alerter-rlbpb\" (UID: \"f8e18877-0f68-4aad-86dc-5fba3a368d76\") " pod="openshift-network-operator/iptables-alerter-rlbpb"
Jul 09 12:02:09.992190 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992092    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5z9f\" (UniqueName: \"kubernetes.io/projected/b994ce8a-e5c9-4828-b792-c9322dd75c26-kube-api-access-r5z9f\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.992190 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992120    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d007f736-c0e5-4cf1-a25c-2dd14dbacb72-agent-certs\") pod \"konnectivity-agent-q4n45\" (UID: \"d007f736-c0e5-4cf1-a25c-2dd14dbacb72\") " pod="kube-system/konnectivity-agent-q4n45"
Jul 09 12:02:09.992190 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992147    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-host\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992190 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992171    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b994ce8a-e5c9-4828-b792-c9322dd75c26-os-release\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.992480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992198    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b994ce8a-e5c9-4828-b792-c9322dd75c26-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.992480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992225    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b994ce8a-e5c9-4828-b792-c9322dd75c26-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.992480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992271    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-kubernetes\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992298    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b994ce8a-e5c9-4828-b792-c9322dd75c26-cnibin\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.992480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992325    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-var-lib-kubelet\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992350    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-sysctl-conf\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992374    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-lib-modules\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992410    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8e18877-0f68-4aad-86dc-5fba3a368d76-host-slash\") pod \"iptables-alerter-rlbpb\" (UID: \"f8e18877-0f68-4aad-86dc-5fba3a368d76\") " pod="openshift-network-operator/iptables-alerter-rlbpb"
Jul 09 12:02:09.992480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992440    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-kubernetes\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992448    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-sysctl-d\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992507    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-tuned\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992538    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fnrs\" (UniqueName: \"kubernetes.io/projected/f8e18877-0f68-4aad-86dc-5fba3a368d76-kube-api-access-9fnrs\") pod \"iptables-alerter-rlbpb\" (UID: \"f8e18877-0f68-4aad-86dc-5fba3a368d76\") " pod="openshift-network-operator/iptables-alerter-rlbpb"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992571    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-tmp\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992589    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-sysctl-d\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992598    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d007f736-c0e5-4cf1-a25c-2dd14dbacb72-konnectivity-ca\") pod \"konnectivity-agent-q4n45\" (UID: \"d007f736-c0e5-4cf1-a25c-2dd14dbacb72\") " pod="kube-system/konnectivity-agent-q4n45"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992630    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-sysconfig\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992657    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-systemd\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992663    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-var-lib-kubelet\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992683    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b994ce8a-e5c9-4828-b792-c9322dd75c26-system-cni-dir\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992720    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b994ce8a-e5c9-4828-b792-c9322dd75c26-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992768    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzdlf\" (UniqueName: \"kubernetes.io/projected/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-kube-api-access-gzdlf\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992794    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-sysctl-conf\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992803    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-modprobe-d\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992468    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b994ce8a-e5c9-4828-b792-c9322dd75c26-os-release\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992841    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-host\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992858    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8e18877-0f68-4aad-86dc-5fba3a368d76-host-slash\") pod \"iptables-alerter-rlbpb\" (UID: \"f8e18877-0f68-4aad-86dc-5fba3a368d76\") " pod="openshift-network-operator/iptables-alerter-rlbpb"
Jul 09 12:02:09.992956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.992937    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-lib-modules\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.993836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.993037    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-modprobe-d\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.993836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.993090    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-sysconfig\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.993836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.993123    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b994ce8a-e5c9-4828-b792-c9322dd75c26-system-cni-dir\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.993836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.993188    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-systemd\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.993836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.993448    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8e18877-0f68-4aad-86dc-5fba3a368d76-iptables-alerter-script\") pod \"iptables-alerter-rlbpb\" (UID: \"f8e18877-0f68-4aad-86dc-5fba3a368d76\") " pod="openshift-network-operator/iptables-alerter-rlbpb"
Jul 09 12:02:09.993836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.993479    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b994ce8a-e5c9-4828-b792-c9322dd75c26-cni-binary-copy\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.993836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.993543    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b994ce8a-e5c9-4828-b792-c9322dd75c26-cnibin\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.993836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.993602    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b994ce8a-e5c9-4828-b792-c9322dd75c26-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.993836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.993665    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-run\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.993836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.993694    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b994ce8a-e5c9-4828-b792-c9322dd75c26-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.993836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.993698    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-sys\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.993836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.993712    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d007f736-c0e5-4cf1-a25c-2dd14dbacb72-konnectivity-ca\") pod \"konnectivity-agent-q4n45\" (UID: \"d007f736-c0e5-4cf1-a25c-2dd14dbacb72\") " pod="kube-system/konnectivity-agent-q4n45"
Jul 09 12:02:09.993836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.993758    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-run\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.993836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.993768    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-sys\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.994750 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.994332    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b994ce8a-e5c9-4828-b792-c9322dd75c26-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:09.996026 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.995984    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-etc-tuned\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.997035 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.996267    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-tmp\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:09.997035 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:09.996507    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d007f736-c0e5-4cf1-a25c-2dd14dbacb72-agent-certs\") pod \"konnectivity-agent-q4n45\" (UID: \"d007f736-c0e5-4cf1-a25c-2dd14dbacb72\") " pod="kube-system/konnectivity-agent-q4n45"
Jul 09 12:02:10.005518 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.005486    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5z9f\" (UniqueName: \"kubernetes.io/projected/b994ce8a-e5c9-4828-b792-c9322dd75c26-kube-api-access-r5z9f\") pod \"multus-additional-cni-plugins-qf2v9\" (UID: \"b994ce8a-e5c9-4828-b792-c9322dd75c26\") " pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:10.005605 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.005576    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fnrs\" (UniqueName: \"kubernetes.io/projected/f8e18877-0f68-4aad-86dc-5fba3a368d76-kube-api-access-9fnrs\") pod \"iptables-alerter-rlbpb\" (UID: \"f8e18877-0f68-4aad-86dc-5fba3a368d76\") " pod="openshift-network-operator/iptables-alerter-rlbpb"
Jul 09 12:02:10.006888 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.006867    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzdlf\" (UniqueName: \"kubernetes.io/projected/1cb4ff2a-cee9-4672-83c9-acfd8d93ebac-kube-api-access-gzdlf\") pod \"tuned-pcvzf\" (UID: \"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac\") " pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:10.091759 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.091658    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq"
Jul 09 12:02:10.097388 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.097364    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zjcb9"
Jul 09 12:02:10.108763 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.108739    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-r5k4m"
Jul 09 12:02:10.115931 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.115908    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mn4b9"
Jul 09 12:02:10.123674 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.123526    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:10.132569 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.132550    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pcvzf"
Jul 09 12:02:10.141421 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.141404    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q4n45"
Jul 09 12:02:10.150159 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.150138    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qf2v9"
Jul 09 12:02:10.157155 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.157138    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rlbpb"
Jul 09 12:02:10.279858 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.279824    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 12:02:10.396086 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.395993    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs\") pod \"network-metrics-daemon-rltxg\" (UID: \"4ac6e98e-e488-4741-aa67-2a6a130291d8\") " pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:10.396267 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.396114    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:10.396267 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:10.396155    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:02:10.396267 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:10.396236    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs podName:4ac6e98e-e488-4741-aa67-2a6a130291d8 nodeName:}" failed. No retries permitted until 2026-07-09 12:02:11.396214478 +0000 UTC m=+4.073743521 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs") pod "network-metrics-daemon-rltxg" (UID: "4ac6e98e-e488-4741-aa67-2a6a130291d8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:02:10.396267 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:10.396252    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 12:02:10.396496 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:10.396321    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret podName:e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b nodeName:}" failed. No retries permitted until 2026-07-09 12:02:11.396301436 +0000 UTC m=+4.073830475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret") pod "global-pull-secret-syncer-p456j" (UID: "e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b") : object "kube-system"/"original-pull-secret" not registered
Jul 09 12:02:10.496942 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.496908    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x28d\" (UniqueName: \"kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d\") pod \"network-check-target-p9wwl\" (UID: \"4afb2515-0939-4e93-9d3d-7f5662d0856e\") " pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:10.497096 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:10.497054    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 12:02:10.497096 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:10.497071    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 12:02:10.497096 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:10.497083    2565 projected.go:196] Error preparing data for projected volume kube-api-access-8x28d for pod openshift-network-diagnostics/network-check-target-p9wwl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:02:10.497228 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:10.497151    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d podName:4afb2515-0939-4e93-9d3d-7f5662d0856e nodeName:}" failed. No retries permitted until 2026-07-09 12:02:11.497129816 +0000 UTC m=+4.174658874 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8x28d" (UniqueName: "kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d") pod "network-check-target-p9wwl" (UID: "4afb2515-0939-4e93-9d3d-7f5662d0856e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:02:10.823736 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.823597    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 11:57:10 +0000 UTC" deadline="2028-03-03 09:11:03.523587737 +0000 UTC"
Jul 09 12:02:10.823736 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.823642    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14469h8m52.699951125s"
Jul 09 12:02:10.888577 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:10.888539    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:10.888787 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:10.888659    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:11.142099 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:11.142067    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8e18877_0f68_4aad_86dc_5fba3a368d76.slice/crio-07cc5a89e9f0be2eacca818c417f93a3a9192109996fe7baedf2a5ce03954318 WatchSource:0}: Error finding container 07cc5a89e9f0be2eacca818c417f93a3a9192109996fe7baedf2a5ce03954318: Status 404 returned error can't find the container with id 07cc5a89e9f0be2eacca818c417f93a3a9192109996fe7baedf2a5ce03954318
Jul 09 12:02:11.144365 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:11.144325    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd007f736_c0e5_4cf1_a25c_2dd14dbacb72.slice/crio-66af16f12e6f500473d12ff74a91adc5d51a8c252dbcc5938109c523a15c9608 WatchSource:0}: Error finding container 66af16f12e6f500473d12ff74a91adc5d51a8c252dbcc5938109c523a15c9608: Status 404 returned error can't find the container with id 66af16f12e6f500473d12ff74a91adc5d51a8c252dbcc5938109c523a15c9608
Jul 09 12:02:11.146259 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:11.145815    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b958117_535b_49f3_b74c_4d809655d7d1.slice/crio-276c85c8f3883ffc8a6d6a97ec83d1d4ea8c24be67995191bdea8204b5cf03f0 WatchSource:0}: Error finding container 276c85c8f3883ffc8a6d6a97ec83d1d4ea8c24be67995191bdea8204b5cf03f0: Status 404 returned error can't find the container with id 276c85c8f3883ffc8a6d6a97ec83d1d4ea8c24be67995191bdea8204b5cf03f0
Jul 09 12:02:11.150461 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:11.150420    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb994ce8a_e5c9_4828_b792_c9322dd75c26.slice/crio-4342cdcd2939b522477644126753db4ee5d903f8049864c4d9c1dc2ef3699010 WatchSource:0}: Error finding container 4342cdcd2939b522477644126753db4ee5d903f8049864c4d9c1dc2ef3699010: Status 404 returned error can't find the container with id 4342cdcd2939b522477644126753db4ee5d903f8049864c4d9c1dc2ef3699010
Jul 09 12:02:11.155279 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:11.155255    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8802ef7_f67c_4aee_81cd_858b426464d9.slice/crio-d3b88b8c2167b67ce1b68d7fb6f2dba329363c750eee5bbcc6f3c422d87ed826 WatchSource:0}: Error finding container d3b88b8c2167b67ce1b68d7fb6f2dba329363c750eee5bbcc6f3c422d87ed826: Status 404 returned error can't find the container with id d3b88b8c2167b67ce1b68d7fb6f2dba329363c750eee5bbcc6f3c422d87ed826
Jul 09 12:02:11.157717 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:11.157636    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ce74bea_4ad5_4a3e_89a3_a04cab4b8ae2.slice/crio-df09512126ea0127ea2c34e83ba8232b34b58f0ec3f9da09e317e2790ae6a1a7 WatchSource:0}: Error finding container df09512126ea0127ea2c34e83ba8232b34b58f0ec3f9da09e317e2790ae6a1a7: Status 404 returned error can't find the container with id df09512126ea0127ea2c34e83ba8232b34b58f0ec3f9da09e317e2790ae6a1a7
Jul 09 12:02:11.404651 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.404322    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs\") pod \"network-metrics-daemon-rltxg\" (UID: \"4ac6e98e-e488-4741-aa67-2a6a130291d8\") " pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:11.404651 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:11.404480    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:02:11.404842 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.404656    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:11.404842 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:11.404731    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs podName:4ac6e98e-e488-4741-aa67-2a6a130291d8 nodeName:}" failed. No retries permitted until 2026-07-09 12:02:13.40469771 +0000 UTC m=+6.082226752 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs") pod "network-metrics-daemon-rltxg" (UID: "4ac6e98e-e488-4741-aa67-2a6a130291d8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:02:11.404842 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:11.404788    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 12:02:11.404951 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:11.404847    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret podName:e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b nodeName:}" failed. No retries permitted until 2026-07-09 12:02:13.404831491 +0000 UTC m=+6.082360529 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret") pod "global-pull-secret-syncer-p456j" (UID: "e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b") : object "kube-system"/"original-pull-secret" not registered
Jul 09 12:02:11.505331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.505291    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x28d\" (UniqueName: \"kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d\") pod \"network-check-target-p9wwl\" (UID: \"4afb2515-0939-4e93-9d3d-7f5662d0856e\") " pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:11.505478 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:11.505438    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 12:02:11.505478 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:11.505455    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 12:02:11.505478 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:11.505465    2565 projected.go:196] Error preparing data for projected volume kube-api-access-8x28d for pod openshift-network-diagnostics/network-check-target-p9wwl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:02:11.505605 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:11.505517    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d podName:4afb2515-0939-4e93-9d3d-7f5662d0856e nodeName:}" failed. No retries permitted until 2026-07-09 12:02:13.505503529 +0000 UTC m=+6.183032566 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8x28d" (UniqueName: "kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d") pod "network-check-target-p9wwl" (UID: "4afb2515-0939-4e93-9d3d-7f5662d0856e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:02:11.825044 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.824785    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 11:57:10 +0000 UTC" deadline="2028-01-08 21:21:09.389621581 +0000 UTC"
Jul 09 12:02:11.825044 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.824854    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13161h18m57.564772451s"
Jul 09 12:02:11.892318 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.891107    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:11.892318 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:11.891257    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:11.892318 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.891838    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:11.892318 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:11.891940    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:11.905421 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.904599    2565 generic.go:356] "Generic (PLEG): container finished" podID="e001a5302fdde64d4d52a11b0c122200" containerID="6585f39a3983488e9f70a9873c0e0dcdfc78c3d44085eadc81ab9753cc82866f" exitCode=0
Jul 09 12:02:11.905421 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.904685    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal" event={"ID":"e001a5302fdde64d4d52a11b0c122200","Type":"ContainerDied","Data":"6585f39a3983488e9f70a9873c0e0dcdfc78c3d44085eadc81ab9753cc82866f"}
Jul 09 12:02:11.911842 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.911809    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r5k4m" event={"ID":"8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2","Type":"ContainerStarted","Data":"df09512126ea0127ea2c34e83ba8232b34b58f0ec3f9da09e317e2790ae6a1a7"}
Jul 09 12:02:11.916036 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.916006    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mn4b9" event={"ID":"b8802ef7-f67c-4aee-81cd-858b426464d9","Type":"ContainerStarted","Data":"d3b88b8c2167b67ce1b68d7fb6f2dba329363c750eee5bbcc6f3c422d87ed826"}
Jul 09 12:02:11.917601 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.917509    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q4n45" event={"ID":"d007f736-c0e5-4cf1-a25c-2dd14dbacb72","Type":"ContainerStarted","Data":"66af16f12e6f500473d12ff74a91adc5d51a8c252dbcc5938109c523a15c9608"}
Jul 09 12:02:11.924019 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.923376    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-94.ec2.internal" event={"ID":"ed9a03f52aaae8a3339d4a55bea80f61","Type":"ContainerStarted","Data":"86564244c2ab8f4fe20e0ce637101c601fc61dbcd67ece1b15d3aff1bc11a2a6"}
Jul 09 12:02:11.928460 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.928428    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq" event={"ID":"7d3e98cc-7393-442a-9b2c-02443b8c3b64","Type":"ContainerStarted","Data":"881b0c4689a495b805a3a37f5b428ecc6839ba718bfcea5da71e5de2386ceccc"}
Jul 09 12:02:11.935092 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.935057    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pcvzf" event={"ID":"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac","Type":"ContainerStarted","Data":"443fbe3f7bebc9d40cb93ede2a2237ee39a3d9666856529784e4c8af503f5005"}
Jul 09 12:02:11.942892 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.942842    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-94.ec2.internal" podStartSLOduration=2.942823293 podStartE2EDuration="2.942823293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:02:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:02:11.942451381 +0000 UTC m=+4.619980444" watchObservedRunningTime="2026-07-09 12:02:11.942823293 +0000 UTC m=+4.620352357"
Jul 09 12:02:11.963093 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.963059    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zjcb9" event={"ID":"760d11e4-c839-41a2-8e35-b4d2566ef960","Type":"ContainerStarted","Data":"b49e2b2b1d27b1c378e6da24976a835a415dae98884e46f15d56a74e357d71af"}
Jul 09 12:02:11.977290 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.977183    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7" event={"ID":"6b958117-535b-49f3-b74c-4d809655d7d1","Type":"ContainerStarted","Data":"276c85c8f3883ffc8a6d6a97ec83d1d4ea8c24be67995191bdea8204b5cf03f0"}
Jul 09 12:02:11.985362 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.985298    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qf2v9" event={"ID":"b994ce8a-e5c9-4828-b792-c9322dd75c26","Type":"ContainerStarted","Data":"4342cdcd2939b522477644126753db4ee5d903f8049864c4d9c1dc2ef3699010"}
Jul 09 12:02:11.992075 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:11.992035    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rlbpb" event={"ID":"f8e18877-0f68-4aad-86dc-5fba3a368d76","Type":"ContainerStarted","Data":"07cc5a89e9f0be2eacca818c417f93a3a9192109996fe7baedf2a5ce03954318"}
Jul 09 12:02:12.888097 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:12.888049    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:12.888628 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:12.888194    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:13.005104 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:13.004873    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal" event={"ID":"e001a5302fdde64d4d52a11b0c122200","Type":"ContainerStarted","Data":"8a43aa1e0fdf9875649c68cfb3452d19999dcb0944309a1c5ea76cfceb9c3a3a"}
Jul 09 12:02:13.423901 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:13.423195    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:13.423901 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:13.423270    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs\") pod \"network-metrics-daemon-rltxg\" (UID: \"4ac6e98e-e488-4741-aa67-2a6a130291d8\") " pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:13.423901 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:13.423445    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:02:13.423901 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:13.423510    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs podName:4ac6e98e-e488-4741-aa67-2a6a130291d8 nodeName:}" failed. No retries permitted until 2026-07-09 12:02:17.423488263 +0000 UTC m=+10.101017309 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs") pod "network-metrics-daemon-rltxg" (UID: "4ac6e98e-e488-4741-aa67-2a6a130291d8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:02:13.423901 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:13.423721    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 12:02:13.423901 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:13.423785    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret podName:e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b nodeName:}" failed. No retries permitted until 2026-07-09 12:02:17.423767144 +0000 UTC m=+10.101296194 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret") pod "global-pull-secret-syncer-p456j" (UID: "e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b") : object "kube-system"/"original-pull-secret" not registered
Jul 09 12:02:13.524740 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:13.524711    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x28d\" (UniqueName: \"kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d\") pod \"network-check-target-p9wwl\" (UID: \"4afb2515-0939-4e93-9d3d-7f5662d0856e\") " pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:13.524972 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:13.524951    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 12:02:13.525044 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:13.524982    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 12:02:13.525044 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:13.524998    2565 projected.go:196] Error preparing data for projected volume kube-api-access-8x28d for pod openshift-network-diagnostics/network-check-target-p9wwl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:02:13.525147 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:13.525055    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d podName:4afb2515-0939-4e93-9d3d-7f5662d0856e nodeName:}" failed. No retries permitted until 2026-07-09 12:02:17.525036242 +0000 UTC m=+10.202565283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8x28d" (UniqueName: "kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d") pod "network-check-target-p9wwl" (UID: "4afb2515-0939-4e93-9d3d-7f5662d0856e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:02:13.888607 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:13.887880    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:13.888607 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:13.888034    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:13.888607 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:13.887880    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:13.888607 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:13.888514    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:14.888395 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:14.887885    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:14.888395 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:14.888034    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:15.887984 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:15.887954    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:15.888449 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:15.888101    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:15.888531 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:15.888466    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:15.888581 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:15.888558    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:16.888232 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:16.888199    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:16.888689 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:16.888335    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:17.461519 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:17.460928    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:17.461519 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:17.460999    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs\") pod \"network-metrics-daemon-rltxg\" (UID: \"4ac6e98e-e488-4741-aa67-2a6a130291d8\") " pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:17.461519 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:17.461121    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:02:17.461519 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:17.461122    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 12:02:17.461519 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:17.461188    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs podName:4ac6e98e-e488-4741-aa67-2a6a130291d8 nodeName:}" failed. No retries permitted until 2026-07-09 12:02:25.461168639 +0000 UTC m=+18.138697689 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs") pod "network-metrics-daemon-rltxg" (UID: "4ac6e98e-e488-4741-aa67-2a6a130291d8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:02:17.461519 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:17.461207    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret podName:e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b nodeName:}" failed. No retries permitted until 2026-07-09 12:02:25.461199856 +0000 UTC m=+18.138728892 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret") pod "global-pull-secret-syncer-p456j" (UID: "e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b") : object "kube-system"/"original-pull-secret" not registered
Jul 09 12:02:17.562150 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:17.562111    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x28d\" (UniqueName: \"kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d\") pod \"network-check-target-p9wwl\" (UID: \"4afb2515-0939-4e93-9d3d-7f5662d0856e\") " pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:17.562331 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:17.562274    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 12:02:17.562331 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:17.562292    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 12:02:17.562331 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:17.562308    2565 projected.go:196] Error preparing data for projected volume kube-api-access-8x28d for pod openshift-network-diagnostics/network-check-target-p9wwl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:02:17.562484 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:17.562354    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d podName:4afb2515-0939-4e93-9d3d-7f5662d0856e nodeName:}" failed. No retries permitted until 2026-07-09 12:02:25.562341806 +0000 UTC m=+18.239870846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8x28d" (UniqueName: "kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d") pod "network-check-target-p9wwl" (UID: "4afb2515-0939-4e93-9d3d-7f5662d0856e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:02:17.888643 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:17.888554    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:17.889088 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:17.888683    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:17.889088 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:17.888738    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:17.889088 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:17.888867    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:18.888266 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:18.888229    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:18.888435 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:18.888371    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:19.888719 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:19.888196    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:19.888719 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:19.888240    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:19.888719 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:19.888345    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:19.888719 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:19.888497    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:20.887885 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:20.887850    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:20.888081 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:20.887972    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:21.888113 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:21.888073    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:21.888113 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:21.888074    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:21.888638 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:21.888219    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:21.888638 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:21.888622    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:22.888304 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:22.888272    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:22.888786 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:22.888397    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:23.888058 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:23.887537    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:23.888058 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:23.887588    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:23.888058 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:23.887683    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:23.888058 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:23.887813    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:24.887657 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:24.887623    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:24.888086 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:24.887752    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:25.520376 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:25.520339    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:25.520528 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:25.520393    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs\") pod \"network-metrics-daemon-rltxg\" (UID: \"4ac6e98e-e488-4741-aa67-2a6a130291d8\") " pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:25.520528 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:25.520485    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:02:25.520528 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:25.520486    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 12:02:25.520645 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:25.520536    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs podName:4ac6e98e-e488-4741-aa67-2a6a130291d8 nodeName:}" failed. No retries permitted until 2026-07-09 12:02:41.520522316 +0000 UTC m=+34.198051351 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs") pod "network-metrics-daemon-rltxg" (UID: "4ac6e98e-e488-4741-aa67-2a6a130291d8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:02:25.520645 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:25.520548    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret podName:e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b nodeName:}" failed. No retries permitted until 2026-07-09 12:02:41.520542561 +0000 UTC m=+34.198071596 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret") pod "global-pull-secret-syncer-p456j" (UID: "e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b") : object "kube-system"/"original-pull-secret" not registered
Jul 09 12:02:25.621389 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:25.621357    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x28d\" (UniqueName: \"kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d\") pod \"network-check-target-p9wwl\" (UID: \"4afb2515-0939-4e93-9d3d-7f5662d0856e\") " pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:25.621543 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:25.621523    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 12:02:25.621632 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:25.621547    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 12:02:25.621632 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:25.621562    2565 projected.go:196] Error preparing data for projected volume kube-api-access-8x28d for pod openshift-network-diagnostics/network-check-target-p9wwl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:02:25.621632 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:25.621614    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d podName:4afb2515-0939-4e93-9d3d-7f5662d0856e nodeName:}" failed. No retries permitted until 2026-07-09 12:02:41.62159918 +0000 UTC m=+34.299128216 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8x28d" (UniqueName: "kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d") pod "network-check-target-p9wwl" (UID: "4afb2515-0939-4e93-9d3d-7f5662d0856e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:02:25.888415 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:25.888340    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:25.888415 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:25.888375    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:25.888916 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:25.888475    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:25.888916 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:25.888587    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:26.888494 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:26.888461    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:26.888970 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:26.888591    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:27.888610 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:27.888569    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:27.889059 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:27.888714    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:27.889059 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:27.888686    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:27.889059 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:27.888785    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:28.887934 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:28.887898    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:28.888092 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:28.888005    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:29.887873 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:29.887842    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:29.888240 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:29.887842    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:29.888240 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:29.887960    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:29.888240 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:29.888021    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:30.888003 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:30.887653    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:30.888898 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:30.888095    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:31.046932 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.046861    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:02:31.047270 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.047248    2565 generic.go:356] "Generic (PLEG): container finished" podID="6b958117-535b-49f3-b74c-4d809655d7d1" containerID="6b3a0e64d649869efb96eafeeb74721ee796b6825a7f9100194c7a4e2e80f46c" exitCode=1
Jul 09 12:02:31.047359 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.047330    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7" event={"ID":"6b958117-535b-49f3-b74c-4d809655d7d1","Type":"ContainerStarted","Data":"a37a4f3e7283cb3b3577e072b17196dd87782748b5e5d4e13d860d6b57dfd70f"}
Jul 09 12:02:31.047423 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.047409    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7" event={"ID":"6b958117-535b-49f3-b74c-4d809655d7d1","Type":"ContainerStarted","Data":"17f326e04d28e308550fa730f9335bfd104ceeb18279c30b5da65720ab98b399"}
Jul 09 12:02:31.047482 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.047461    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7" event={"ID":"6b958117-535b-49f3-b74c-4d809655d7d1","Type":"ContainerStarted","Data":"49cd1d58a1a5be022b804fd041c7d1eb8747f1116cc87df7b9774bfc41811deb"}
Jul 09 12:02:31.047536 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.047502    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7" event={"ID":"6b958117-535b-49f3-b74c-4d809655d7d1","Type":"ContainerStarted","Data":"edb246e28c85de8c38aeff48305fe189b6cd1e34ae5675f5692c78981ff26621"}
Jul 09 12:02:31.047584 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.047551    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7" event={"ID":"6b958117-535b-49f3-b74c-4d809655d7d1","Type":"ContainerDied","Data":"6b3a0e64d649869efb96eafeeb74721ee796b6825a7f9100194c7a4e2e80f46c"}
Jul 09 12:02:31.047633 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.047595    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7" event={"ID":"6b958117-535b-49f3-b74c-4d809655d7d1","Type":"ContainerStarted","Data":"c538dcb83ccd7aecf9bd3fb72f850a72875503cd0f3565fec1c4487946996a26"}
Jul 09 12:02:31.048897 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.048872    2565 generic.go:356] "Generic (PLEG): container finished" podID="b994ce8a-e5c9-4828-b792-c9322dd75c26" containerID="7c582caf7a57a6ca676ce533178b4fa35fc1135677a581c2c197859996a85538" exitCode=0
Jul 09 12:02:31.049008 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.048930    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qf2v9" event={"ID":"b994ce8a-e5c9-4828-b792-c9322dd75c26","Type":"ContainerDied","Data":"7c582caf7a57a6ca676ce533178b4fa35fc1135677a581c2c197859996a85538"}
Jul 09 12:02:31.050674 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.050587    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-r5k4m" event={"ID":"8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2","Type":"ContainerStarted","Data":"da45183d0b9a02dfdef9d14671012d80dcb6b64ba49a86e1084ca4d24a189c69"}
Jul 09 12:02:31.052515 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.052470    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mn4b9" event={"ID":"b8802ef7-f67c-4aee-81cd-858b426464d9","Type":"ContainerStarted","Data":"043e885da13ca71a93a6d7d44befb126cb4d56e8d94d49683b968ebc805a8f2f"}
Jul 09 12:02:31.054076 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.054052    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q4n45" event={"ID":"d007f736-c0e5-4cf1-a25c-2dd14dbacb72","Type":"ContainerStarted","Data":"94bc23ce7f42d18481ccefe0ff8e2e421a6b8ad53c91bdfb5893ff83ee73ea80"}
Jul 09 12:02:31.055663 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.055643    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq" event={"ID":"7d3e98cc-7393-442a-9b2c-02443b8c3b64","Type":"ContainerStarted","Data":"9045c45bd169ee945c55c0e226fef49825647e5ed663102288b35ab5f1ca77cb"}
Jul 09 12:02:31.057282 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.057236    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pcvzf" event={"ID":"1cb4ff2a-cee9-4672-83c9-acfd8d93ebac","Type":"ContainerStarted","Data":"bed8690d0b9a8456a99d97bad9968544732bc0bed70a0cf4096bfafa7a731865"}
Jul 09 12:02:31.059075 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.059052    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zjcb9" event={"ID":"760d11e4-c839-41a2-8e35-b4d2566ef960","Type":"ContainerStarted","Data":"8d539d6bb37fd6fc7500f5b218305e4fdb9ee31c134345149e793a17028101b1"}
Jul 09 12:02:31.074192 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.074158    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-94.ec2.internal" podStartSLOduration=22.074146943 podStartE2EDuration="22.074146943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:02:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:02:13.022281393 +0000 UTC m=+5.699810455" watchObservedRunningTime="2026-07-09 12:02:31.074146943 +0000 UTC m=+23.751676001"
Jul 09 12:02:31.109508 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.109459    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pcvzf" podStartSLOduration=4.212134259 podStartE2EDuration="23.109442417s" totalImagesPullingTime="18.897308158s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:02:31.095421207 +0000 UTC m=+23.772950282" watchObservedRunningTime="2026-07-09 12:02:31.109442417 +0000 UTC m=+23.786971455"
Jul 09 12:02:31.110203 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.110170    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-q4n45" podStartSLOduration=4.223891712 podStartE2EDuration="23.110155777s" totalImagesPullingTime="18.886264065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:02:31.109239407 +0000 UTC m=+23.786768469" watchObservedRunningTime="2026-07-09 12:02:31.110155777 +0000 UTC m=+23.787684836"
Jul 09 12:02:31.124071 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.123952    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zjcb9" podStartSLOduration=4.225530392 podStartE2EDuration="23.123936223s" totalImagesPullingTime="18.898405831s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:02:31.12377747 +0000 UTC m=+23.801306528" watchObservedRunningTime="2026-07-09 12:02:31.123936223 +0000 UTC m=+23.801465282"
Jul 09 12:02:31.138906 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.138870    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-r5k4m" podStartSLOduration=4.263938199 podStartE2EDuration="23.13885882s" totalImagesPullingTime="18.874920621s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:02:31.138368269 +0000 UTC m=+23.815897327" watchObservedRunningTime="2026-07-09 12:02:31.13885882 +0000 UTC m=+23.816387879"
Jul 09 12:02:31.156072 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.156035    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mn4b9" podStartSLOduration=3.91531055 podStartE2EDuration="23.156022377s" totalImagesPullingTime="19.240711827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:02:31.155502231 +0000 UTC m=+23.833031289" watchObservedRunningTime="2026-07-09 12:02:31.156022377 +0000 UTC m=+23.833551433"
Jul 09 12:02:31.693943 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.693921    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 12:02:31.859813 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.859648    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T12:02:31.693940011Z","UUID":"6aebc9cb-1273-4aeb-a9a7-973d0dfb2d56","Handler":null,"Name":"","Endpoint":""}
Jul 09 12:02:31.861012 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.860997    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 12:02:31.861112 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.861029    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 12:02:31.887746 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.887719    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:31.887850 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:31.887719    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:31.887850 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:31.887827    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:31.887919 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:31.887891    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:32.062817 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:32.062781    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq" event={"ID":"7d3e98cc-7393-442a-9b2c-02443b8c3b64","Type":"ContainerStarted","Data":"160c16c0c20845f539821400c8163b0592a056b6fb54a31d8fe0483affae8667"}
Jul 09 12:02:32.064242 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:32.064214    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rlbpb" event={"ID":"f8e18877-0f68-4aad-86dc-5fba3a368d76","Type":"ContainerStarted","Data":"5b390ba3e9b1345524c4a2ec1f24d5bd77f13ac59e9adbfe48f8da2336a09c1a"}
Jul 09 12:02:32.080197 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:32.080161    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rlbpb" podStartSLOduration=5.175111519 podStartE2EDuration="24.080148129s" totalImagesPullingTime="18.90503661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:02:32.07992855 +0000 UTC m=+24.757457608" watchObservedRunningTime="2026-07-09 12:02:32.080148129 +0000 UTC m=+24.757677187"
Jul 09 12:02:32.833747 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:32.833504    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-q4n45"
Jul 09 12:02:32.834024 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:32.834009    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-q4n45"
Jul 09 12:02:32.888404 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:32.888383    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:32.888502 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:32.888484    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:33.068480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:33.068446    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq" event={"ID":"7d3e98cc-7393-442a-9b2c-02443b8c3b64","Type":"ContainerStarted","Data":"f1978f12e6df33a6fd631f40db0d4aae786bd3c291094f5e604728d37a77af31"}
Jul 09 12:02:33.071286 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:33.071259    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:02:33.071597 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:33.071577    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7" event={"ID":"6b958117-535b-49f3-b74c-4d809655d7d1","Type":"ContainerStarted","Data":"f21a6d845c9b552bdb4207789c2da2b37e668b3e8f184ecc1d072fb816770c25"}
Jul 09 12:02:33.087026 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:33.086946    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxckq" podStartSLOduration=4.095749326 podStartE2EDuration="25.08693453s" totalImagesPullingTime="20.991185204s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:02:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:02:33.086493338 +0000 UTC m=+25.764022397" watchObservedRunningTime="2026-07-09 12:02:33.08693453 +0000 UTC m=+25.764463588"
Jul 09 12:02:33.165760 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:33.165735    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-q4n45"
Jul 09 12:02:33.166342 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:33.166321    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-q4n45"
Jul 09 12:02:33.888224 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:33.888190    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:33.888369 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:33.888331    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:33.888432 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:33.888375    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:33.888478 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:33.888459    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:34.887840 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:34.887794    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:34.888332 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:34.887934    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:35.080013 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:35.079996    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:02:35.080424 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:35.080398    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7" event={"ID":"6b958117-535b-49f3-b74c-4d809655d7d1","Type":"ContainerStarted","Data":"7cd8d90a2c103a0720bfb27451e90a6cd686f69d6b586075cc022501fccd7c9f"}
Jul 09 12:02:35.080677 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:35.080653    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:35.080813 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:35.080768    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:35.081049 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:35.080999    2565 scope.go:117] "RemoveContainer" containerID="6b3a0e64d649869efb96eafeeb74721ee796b6825a7f9100194c7a4e2e80f46c"
Jul 09 12:02:35.100097 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:35.099884    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:35.887905 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:35.887865    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:35.888609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:35.887865    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:35.888609 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:35.887994    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:35.888609 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:35.888040    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:36.088382 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:36.088360    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:02:36.088721 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:36.088682    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7" event={"ID":"6b958117-535b-49f3-b74c-4d809655d7d1","Type":"ContainerStarted","Data":"277b30d3cf9797bbce2b61a73e827aededb2653d4cda62a796ef6912fa634adc"}
Jul 09 12:02:36.088944 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:36.088926    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:36.104332 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:36.104311    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:02:36.118759 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:36.118679    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7" podStartSLOduration=9.145628633 podStartE2EDuration="28.118667403s" totalImagesPullingTime="18.97303877s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:02:36.117137716 +0000 UTC m=+28.794666773" watchObservedRunningTime="2026-07-09 12:02:36.118667403 +0000 UTC m=+28.796196462"
Jul 09 12:02:36.888242 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:36.888215    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:36.888644 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:36.888329    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:37.051846 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:37.051816    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p9wwl"]
Jul 09 12:02:37.052018 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:37.051953    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:37.052090 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:37.052063    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:37.056671 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:37.056596    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p456j"]
Jul 09 12:02:37.057389 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:37.057356    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rltxg"]
Jul 09 12:02:37.057505 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:37.057489    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:37.057655 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:37.057611    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:37.091233 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:37.091168    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:37.091364 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:37.091276    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:38.887907 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:38.887739    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:38.888295 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:38.887774    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:38.888295 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:38.887976    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:38.888295 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:38.888053    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:38.888295 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:38.887774    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:38.888295 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:38.888118    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:39.098146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:39.098113    2565 generic.go:356] "Generic (PLEG): container finished" podID="b994ce8a-e5c9-4828-b792-c9322dd75c26" containerID="6e52aba895b96dbbd5779407094fc5d0de91e51f9fd71ea05626b34d04a02fae" exitCode=0
Jul 09 12:02:39.098301 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:39.098175    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qf2v9" event={"ID":"b994ce8a-e5c9-4828-b792-c9322dd75c26","Type":"ContainerDied","Data":"6e52aba895b96dbbd5779407094fc5d0de91e51f9fd71ea05626b34d04a02fae"}
Jul 09 12:02:40.887950 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:40.887912    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:40.888590 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:40.887962    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:40.888590 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:40.888016    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:40.888590 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:40.888034    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:40.888590 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:40.888091    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:40.888590 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:40.888151    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:41.105077 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:41.105045    2565 generic.go:356] "Generic (PLEG): container finished" podID="b994ce8a-e5c9-4828-b792-c9322dd75c26" containerID="c949e3ff649cfb2ed1f01b383831b63bb71e776bd7b09bc2ff9a74cd6930b388" exitCode=0
Jul 09 12:02:41.105223 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:41.105096    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qf2v9" event={"ID":"b994ce8a-e5c9-4828-b792-c9322dd75c26","Type":"ContainerDied","Data":"c949e3ff649cfb2ed1f01b383831b63bb71e776bd7b09bc2ff9a74cd6930b388"}
Jul 09 12:02:41.532511 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:41.532479    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:41.532717 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:41.532536    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs\") pod \"network-metrics-daemon-rltxg\" (UID: \"4ac6e98e-e488-4741-aa67-2a6a130291d8\") " pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:41.532717 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:41.532646    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 12:02:41.532717 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:41.532681    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:02:41.532870 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:41.532723    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret podName:e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b nodeName:}" failed. No retries permitted until 2026-07-09 12:03:13.532687451 +0000 UTC m=+66.210216493 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret") pod "global-pull-secret-syncer-p456j" (UID: "e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b") : object "kube-system"/"original-pull-secret" not registered
Jul 09 12:02:41.532870 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:41.532763    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs podName:4ac6e98e-e488-4741-aa67-2a6a130291d8 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:13.532742804 +0000 UTC m=+66.210271840 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs") pod "network-metrics-daemon-rltxg" (UID: "4ac6e98e-e488-4741-aa67-2a6a130291d8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 12:02:41.633240 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:41.633210    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x28d\" (UniqueName: \"kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d\") pod \"network-check-target-p9wwl\" (UID: \"4afb2515-0939-4e93-9d3d-7f5662d0856e\") " pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:41.633392 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:41.633371    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 12:02:41.633392 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:41.633392    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 12:02:41.633470 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:41.633402    2565 projected.go:196] Error preparing data for projected volume kube-api-access-8x28d for pod openshift-network-diagnostics/network-check-target-p9wwl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:02:41.633470 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:41.633450    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d podName:4afb2515-0939-4e93-9d3d-7f5662d0856e nodeName:}" failed. No retries permitted until 2026-07-09 12:03:13.633438036 +0000 UTC m=+66.310967072 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8x28d" (UniqueName: "kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d") pod "network-check-target-p9wwl" (UID: "4afb2515-0939-4e93-9d3d-7f5662d0856e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 12:02:42.888247 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:42.888065    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:42.888633 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:42.888065    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:42.888633 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:42.888368    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rltxg" podUID="4ac6e98e-e488-4741-aa67-2a6a130291d8"
Jul 09 12:02:42.888633 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:42.888065    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:42.888633 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:42.888385    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9wwl" podUID="4afb2515-0939-4e93-9d3d-7f5662d0856e"
Jul 09 12:02:42.888633 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:42.888460    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p456j" podUID="e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b"
Jul 09 12:02:43.111322 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.111196    2565 generic.go:356] "Generic (PLEG): container finished" podID="b994ce8a-e5c9-4828-b792-c9322dd75c26" containerID="5e9d8589f594ba62a606084091b883c0c646713e4954b7f00e2a27a0e76960d6" exitCode=0
Jul 09 12:02:43.111322 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.111248    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qf2v9" event={"ID":"b994ce8a-e5c9-4828-b792-c9322dd75c26","Type":"ContainerDied","Data":"5e9d8589f594ba62a606084091b883c0c646713e4954b7f00e2a27a0e76960d6"}
Jul 09 12:02:43.131015 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.130996    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-94.ec2.internal" event="NodeReady"
Jul 09 12:02:43.131109 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.131100    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 12:02:43.175668 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.175646    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xw2dj"]
Jul 09 12:02:43.177807 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.177791    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:43.181490 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.181349    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 12:02:43.181490 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.181371    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 12:02:43.181490 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.181380    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mzdcd\""
Jul 09 12:02:43.181829 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.181814    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dcmlh"]
Jul 09 12:02:43.183943 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.183923    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dcmlh"
Jul 09 12:02:43.187606 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.187579    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 12:02:43.187730 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.187644    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 12:02:43.187730 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.187655    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 12:02:43.187845 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.187726    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7t5w9\""
Jul 09 12:02:43.191659 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.191639    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xw2dj"]
Jul 09 12:02:43.196135 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.196114    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dcmlh"]
Jul 09 12:02:43.245099 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.245072    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm4tj\" (UniqueName: \"kubernetes.io/projected/f77b9023-6512-43a6-8d89-2ea512bc8c10-kube-api-access-xm4tj\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:43.245236 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.245123    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77b9023-6512-43a6-8d89-2ea512bc8c10-tmp-dir\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:43.245236 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.245208    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f77b9023-6512-43a6-8d89-2ea512bc8c10-config-volume\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:43.245339 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.245279    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:43.346520 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.346489    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:43.346627 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.346526    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm4tj\" (UniqueName: \"kubernetes.io/projected/f77b9023-6512-43a6-8d89-2ea512bc8c10-kube-api-access-xm4tj\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:43.346664 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:43.346623    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 12:02:43.346695 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:43.346675    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls podName:f77b9023-6512-43a6-8d89-2ea512bc8c10 nodeName:}" failed. No retries permitted until 2026-07-09 12:02:43.84666017 +0000 UTC m=+36.524189209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls") pod "dns-default-xw2dj" (UID: "f77b9023-6512-43a6-8d89-2ea512bc8c10") : secret "dns-default-metrics-tls" not found
Jul 09 12:02:43.346695 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.346624    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f68292df-cc6d-4528-bac8-3a6badb20ed2-cert\") pod \"ingress-canary-dcmlh\" (UID: \"f68292df-cc6d-4528-bac8-3a6badb20ed2\") " pod="openshift-ingress-canary/ingress-canary-dcmlh"
Jul 09 12:02:43.346825 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.346736    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77b9023-6512-43a6-8d89-2ea512bc8c10-tmp-dir\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:43.346825 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.346767    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk6d2\" (UniqueName: \"kubernetes.io/projected/f68292df-cc6d-4528-bac8-3a6badb20ed2-kube-api-access-tk6d2\") pod \"ingress-canary-dcmlh\" (UID: \"f68292df-cc6d-4528-bac8-3a6badb20ed2\") " pod="openshift-ingress-canary/ingress-canary-dcmlh"
Jul 09 12:02:43.346825 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.346805    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f77b9023-6512-43a6-8d89-2ea512bc8c10-config-volume\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:43.347050 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.347035    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f77b9023-6512-43a6-8d89-2ea512bc8c10-tmp-dir\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:43.347237 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.347223    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f77b9023-6512-43a6-8d89-2ea512bc8c10-config-volume\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:43.356346 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.356326    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm4tj\" (UniqueName: \"kubernetes.io/projected/f77b9023-6512-43a6-8d89-2ea512bc8c10-kube-api-access-xm4tj\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:43.448081 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.448007    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f68292df-cc6d-4528-bac8-3a6badb20ed2-cert\") pod \"ingress-canary-dcmlh\" (UID: \"f68292df-cc6d-4528-bac8-3a6badb20ed2\") " pod="openshift-ingress-canary/ingress-canary-dcmlh"
Jul 09 12:02:43.448081 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.448047    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk6d2\" (UniqueName: \"kubernetes.io/projected/f68292df-cc6d-4528-bac8-3a6badb20ed2-kube-api-access-tk6d2\") pod \"ingress-canary-dcmlh\" (UID: \"f68292df-cc6d-4528-bac8-3a6badb20ed2\") " pod="openshift-ingress-canary/ingress-canary-dcmlh"
Jul 09 12:02:43.450442 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.450415    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f68292df-cc6d-4528-bac8-3a6badb20ed2-cert\") pod \"ingress-canary-dcmlh\" (UID: \"f68292df-cc6d-4528-bac8-3a6badb20ed2\") " pod="openshift-ingress-canary/ingress-canary-dcmlh"
Jul 09 12:02:43.455524 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.455505    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk6d2\" (UniqueName: \"kubernetes.io/projected/f68292df-cc6d-4528-bac8-3a6badb20ed2-kube-api-access-tk6d2\") pod \"ingress-canary-dcmlh\" (UID: \"f68292df-cc6d-4528-bac8-3a6badb20ed2\") " pod="openshift-ingress-canary/ingress-canary-dcmlh"
Jul 09 12:02:43.496465 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.496425    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dcmlh"
Jul 09 12:02:43.627386 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.627357    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dcmlh"]
Jul 09 12:02:43.631118 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:02:43.631091    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf68292df_cc6d_4528_bac8_3a6badb20ed2.slice/crio-78a2755ecb452e96fd7ae58339257747c8e30d966567989a45d0e525235b6ba9 WatchSource:0}: Error finding container 78a2755ecb452e96fd7ae58339257747c8e30d966567989a45d0e525235b6ba9: Status 404 returned error can't find the container with id 78a2755ecb452e96fd7ae58339257747c8e30d966567989a45d0e525235b6ba9
Jul 09 12:02:43.850480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:43.850440    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:43.850623 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:43.850569    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 12:02:43.850685 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:43.850640    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls podName:f77b9023-6512-43a6-8d89-2ea512bc8c10 nodeName:}" failed. No retries permitted until 2026-07-09 12:02:44.850623136 +0000 UTC m=+37.528152172 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls") pod "dns-default-xw2dj" (UID: "f77b9023-6512-43a6-8d89-2ea512bc8c10") : secret "dns-default-metrics-tls" not found
Jul 09 12:02:44.115766 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:44.115648    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dcmlh" event={"ID":"f68292df-cc6d-4528-bac8-3a6badb20ed2","Type":"ContainerStarted","Data":"78a2755ecb452e96fd7ae58339257747c8e30d966567989a45d0e525235b6ba9"}
Jul 09 12:02:44.859553 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:44.859510    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:44.859764 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:44.859734    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 12:02:44.859839 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:44.859809    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls podName:f77b9023-6512-43a6-8d89-2ea512bc8c10 nodeName:}" failed. No retries permitted until 2026-07-09 12:02:46.859787888 +0000 UTC m=+39.537316927 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls") pod "dns-default-xw2dj" (UID: "f77b9023-6512-43a6-8d89-2ea512bc8c10") : secret "dns-default-metrics-tls" not found
Jul 09 12:02:44.888056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:44.888025    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:02:44.888226 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:44.888060    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:02:44.888226 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:44.888060    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:02:44.891065 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:44.891042    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 12:02:44.892283 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:44.892256    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 12:02:44.892481 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:44.892462    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 12:02:44.892650 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:44.892637    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cpf42\""
Jul 09 12:02:44.893357 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:44.893341    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 12:02:44.900899 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:44.900880    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-62g6t\""
Jul 09 12:02:46.875787 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:46.875540    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:46.875787 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:46.875731    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 12:02:46.876332 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:46.875809    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls podName:f77b9023-6512-43a6-8d89-2ea512bc8c10 nodeName:}" failed. No retries permitted until 2026-07-09 12:02:50.875788448 +0000 UTC m=+43.553317485 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls") pod "dns-default-xw2dj" (UID: "f77b9023-6512-43a6-8d89-2ea512bc8c10") : secret "dns-default-metrics-tls" not found
Jul 09 12:02:47.125470 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:47.125429    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dcmlh" event={"ID":"f68292df-cc6d-4528-bac8-3a6badb20ed2","Type":"ContainerStarted","Data":"86cea7574729d977674dfb3dfff6b2fef83584cf368ee5bed08df63904a0933e"}
Jul 09 12:02:47.144357 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:47.144260    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dcmlh" podStartSLOduration=1.467780592 podStartE2EDuration="4.144245953s" totalImagesPullingTime="2.676465361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:02:47.143935023 +0000 UTC m=+39.821464089" watchObservedRunningTime="2026-07-09 12:02:47.144245953 +0000 UTC m=+39.821775010"
Jul 09 12:02:49.132513 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:49.132481    2565 generic.go:356] "Generic (PLEG): container finished" podID="b994ce8a-e5c9-4828-b792-c9322dd75c26" containerID="ba7120d1bb6557e5e819c416f6e03151b9f454b207680d0f23332006cd22c310" exitCode=0
Jul 09 12:02:49.133130 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:49.132526    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qf2v9" event={"ID":"b994ce8a-e5c9-4828-b792-c9322dd75c26","Type":"ContainerDied","Data":"ba7120d1bb6557e5e819c416f6e03151b9f454b207680d0f23332006cd22c310"}
Jul 09 12:02:50.137807 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:50.137771    2565 generic.go:356] "Generic (PLEG): container finished" podID="b994ce8a-e5c9-4828-b792-c9322dd75c26" containerID="24a33655b06cacfe6648cb72e7a7530aec71a0054e7d09b8e84a6b2e5aaf378f" exitCode=0
Jul 09 12:02:50.137807 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:50.137810    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qf2v9" event={"ID":"b994ce8a-e5c9-4828-b792-c9322dd75c26","Type":"ContainerDied","Data":"24a33655b06cacfe6648cb72e7a7530aec71a0054e7d09b8e84a6b2e5aaf378f"}
Jul 09 12:02:50.904605 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:50.904555    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:50.904800 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:50.904718    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 12:02:50.904800 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:50.904778    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls podName:f77b9023-6512-43a6-8d89-2ea512bc8c10 nodeName:}" failed. No retries permitted until 2026-07-09 12:02:58.904763906 +0000 UTC m=+51.582292942 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls") pod "dns-default-xw2dj" (UID: "f77b9023-6512-43a6-8d89-2ea512bc8c10") : secret "dns-default-metrics-tls" not found
Jul 09 12:02:51.143800 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:51.143765    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qf2v9" event={"ID":"b994ce8a-e5c9-4828-b792-c9322dd75c26","Type":"ContainerStarted","Data":"07a7bcb511dc63f5eb3bf657c6fadc700e14e7b9d3aafca2567f85719dd30ff0"}
Jul 09 12:02:51.168519 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:51.168431    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qf2v9" podStartSLOduration=9.404457401 podStartE2EDuration="43.168416618s" totalImagesPullingTime="33.763959217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:02:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:02:51.166724191 +0000 UTC m=+43.844253243" watchObservedRunningTime="2026-07-09 12:02:51.168416618 +0000 UTC m=+43.845945677"
Jul 09 12:02:58.960846 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:02:58.960809    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:02:58.961402 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:58.960942    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 12:02:58.961402 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:02:58.961013    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls podName:f77b9023-6512-43a6-8d89-2ea512bc8c10 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:14.96099403 +0000 UTC m=+67.638523083 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls") pod "dns-default-xw2dj" (UID: "f77b9023-6512-43a6-8d89-2ea512bc8c10") : secret "dns-default-metrics-tls" not found
Jul 09 12:03:08.104057 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:08.104021    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l2hg7"
Jul 09 12:03:12.816518 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.816479    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"]
Jul 09 12:03:12.838537 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.838512    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"]
Jul 09 12:03:12.838673 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.838599    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:12.841489 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.841470    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 09 12:03:12.841605 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.841543    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 12:03:12.842803 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.842783    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 12:03:12.842997 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.842974    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nds67\""
Jul 09 12:03:12.843498 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.843482    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 09 12:03:12.918809 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.918781    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz"]
Jul 09 12:03:12.935958 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.935934    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xl7np"]
Jul 09 12:03:12.936058 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.935974    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz"
Jul 09 12:03:12.944386 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.944362    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 09 12:03:12.944568 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.944546    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 09 12:03:12.944881 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.944860    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wtzqk\""
Jul 09 12:03:12.944881 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.944880    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 09 12:03:12.945053 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.944881    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 09 12:03:12.948572 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.948555    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"]
Jul 09 12:03:12.948953 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.948729    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xl7np"
Jul 09 12:03:12.951756 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.951737    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 09 12:03:12.952422 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.952405    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 09 12:03:12.952810 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.952795    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9s645\""
Jul 09 12:03:12.957890 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.957871    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ba5787df-e1ac-4ae8-af04-901fc636f023-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:12.957967 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.957902    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:12.957967 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.957936    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw2mr\" (UniqueName: \"kubernetes.io/projected/ba5787df-e1ac-4ae8-af04-901fc636f023-kube-api-access-fw2mr\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:12.963578 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.963561    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-k662b"]
Jul 09 12:03:12.963744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.963725    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"
Jul 09 12:03:12.966546 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.966528    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 09 12:03:12.966626 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.966542    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 09 12:03:12.966626 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.966603    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pd6hz\""
Jul 09 12:03:12.966626 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.966550    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 09 12:03:12.982111 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.982094    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xl7np"]
Jul 09 12:03:12.982200 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.982122    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz"]
Jul 09 12:03:12.982200 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.982135    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-k662b"]
Jul 09 12:03:12.982200 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.982151    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"]
Jul 09 12:03:12.982323 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.982231    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:12.984847 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.984831    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mcs5c\""
Jul 09 12:03:12.985034 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.985001    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 09 12:03:12.985034 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.985013    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 09 12:03:12.985159 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.985107    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 12:03:12.985194 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.985107    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 12:03:12.990258 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:12.990241    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 09 12:03:13.016345 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.016326    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-zjkp8"]
Jul 09 12:03:13.032855 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.032831    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-787c4f4479-255zf"]
Jul 09 12:03:13.032945 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.032903    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zjkp8"
Jul 09 12:03:13.036064 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.036048    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-tpb2n\""
Jul 09 12:03:13.055290 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.055272    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-zjkp8"]
Jul 09 12:03:13.055372 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.055299    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-787c4f4479-255zf"]
Jul 09 12:03:13.055372 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.055364    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.058520 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.058502    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 09 12:03:13.058834 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.058813    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 09 12:03:13.058970 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.058956    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b9d000a-c712-4679-8044-c0a4f18920f0-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.059015 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.058984    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-certificates\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.059049 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059018    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ba5787df-e1ac-4ae8-af04-901fc636f023-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:13.059082 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059060    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmw4j\" (UniqueName: \"kubernetes.io/projected/24691895-ba59-41b7-8cb0-eb97b4708875-kube-api-access-lmw4j\") pod \"service-ca-operator-5d46d9ffd4-8g6sz\" (UID: \"24691895-ba59-41b7-8cb0-eb97b4708875\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz"
Jul 09 12:03:13.059082 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059072    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-r2nb5\""
Jul 09 12:03:13.059193 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059096    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-trusted-ca\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.059193 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059125    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zzt8\" (UniqueName: \"kubernetes.io/projected/3b9d000a-c712-4679-8044-c0a4f18920f0-kube-api-access-6zzt8\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.059193 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059159    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-image-registry-private-configuration\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.059337 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059281    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 09 12:03:13.059337 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059290    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dmclb\" (UID: \"165ea525-4d56-41f2-85b8-772b6abe82ff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"
Jul 09 12:03:13.059458 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059338    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b9d000a-c712-4679-8044-c0a4f18920f0-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.059458 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059362    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-bound-sa-token\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.059458 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059397    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24691895-ba59-41b7-8cb0-eb97b4708875-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-8g6sz\" (UID: \"24691895-ba59-41b7-8cb0-eb97b4708875\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz"
Jul 09 12:03:13.059458 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059417    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltx9j\" (UniqueName: \"kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j\") pod \"cluster-samples-operator-56fcf86bf-dmclb\" (UID: \"165ea525-4d56-41f2-85b8-772b6abe82ff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"
Jul 09 12:03:13.059458 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059444    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3b9d000a-c712-4679-8044-c0a4f18920f0-snapshots\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.059687 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059461    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/24691895-ba59-41b7-8cb0-eb97b4708875-config\") pod \"service-ca-operator-5d46d9ffd4-8g6sz\" (UID: \"24691895-ba59-41b7-8cb0-eb97b4708875\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz"
Jul 09 12:03:13.059687 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059476    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plr42\" (UniqueName: \"kubernetes.io/projected/a7de5580-08d4-449f-9f76-7fab78680c74-kube-api-access-plr42\") pod \"network-check-source-989f8b8dc-zjkp8\" (UID: \"a7de5580-08d4-449f-9f76-7fab78680c74\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zjkp8"
Jul 09 12:03:13.059687 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059494    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:13.059687 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059530    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-ca-trust-extracted\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.059687 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.059553    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 12:03:13.059687 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059565    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-installation-pull-secrets\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.059687 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.059597    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls podName:ba5787df-e1ac-4ae8-af04-901fc636f023 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:13.559584477 +0000 UTC m=+66.237113512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xnp29" (UID: "ba5787df-e1ac-4ae8-af04-901fc636f023") : secret "cluster-monitoring-operator-tls" not found
Jul 09 12:03:13.059687 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059609    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.059687 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059630    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw2mr\" (UniqueName: \"kubernetes.io/projected/ba5787df-e1ac-4ae8-af04-901fc636f023-kube-api-access-fw2mr\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:13.059687 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059639    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ba5787df-e1ac-4ae8-af04-901fc636f023-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:13.060193 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.059674    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnmwg\" (UniqueName: \"kubernetes.io/projected/d4216769-1d00-4c9b-b986-a577ad840711-kube-api-access-wnmwg\") pod \"volume-data-source-validator-5d456fdbcd-xl7np\" (UID: \"d4216769-1d00-4c9b-b986-a577ad840711\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xl7np"
Jul 09 12:03:13.060193 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.060146    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46x6k\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-kube-api-access-46x6k\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.060281 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.060263    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b9d000a-c712-4679-8044-c0a4f18920f0-tmp\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.060331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.060300    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3b9d000a-c712-4679-8044-c0a4f18920f0-serving-cert\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.066431 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.066410    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 09 12:03:13.075446 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.075396    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw2mr\" (UniqueName: \"kubernetes.io/projected/ba5787df-e1ac-4ae8-af04-901fc636f023-kube-api-access-fw2mr\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:13.161072 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161042    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnmwg\" (UniqueName: \"kubernetes.io/projected/d4216769-1d00-4c9b-b986-a577ad840711-kube-api-access-wnmwg\") pod \"volume-data-source-validator-5d456fdbcd-xl7np\" (UID: \"d4216769-1d00-4c9b-b986-a577ad840711\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xl7np"
Jul 09 12:03:13.161072 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161074    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46x6k\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-kube-api-access-46x6k\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.161248 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161113    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b9d000a-c712-4679-8044-c0a4f18920f0-tmp\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.161307 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161240    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3b9d000a-c712-4679-8044-c0a4f18920f0-serving-cert\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.161307 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161289    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b9d000a-c712-4679-8044-c0a4f18920f0-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.161410 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161315    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-certificates\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.161410 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161357    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmw4j\" (UniqueName: \"kubernetes.io/projected/24691895-ba59-41b7-8cb0-eb97b4708875-kube-api-access-lmw4j\") pod \"service-ca-operator-5d46d9ffd4-8g6sz\" (UID: \"24691895-ba59-41b7-8cb0-eb97b4708875\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz"
Jul 09 12:03:13.161410 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161386    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-trusted-ca\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.161562 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161465    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b9d000a-c712-4679-8044-c0a4f18920f0-tmp\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.161562 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161519    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zzt8\" (UniqueName: \"kubernetes.io/projected/3b9d000a-c712-4679-8044-c0a4f18920f0-kube-api-access-6zzt8\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.161670 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161566    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-image-registry-private-configuration\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.161670 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161630    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dmclb\" (UID: \"165ea525-4d56-41f2-85b8-772b6abe82ff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"
Jul 09 12:03:13.161670 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161657    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b9d000a-c712-4679-8044-c0a4f18920f0-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.161846 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161674    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-bound-sa-token\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.161846 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161721    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24691895-ba59-41b7-8cb0-eb97b4708875-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-8g6sz\" (UID: \"24691895-ba59-41b7-8cb0-eb97b4708875\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz"
Jul 09 12:03:13.161846 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161749    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltx9j\" (UniqueName: \"kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j\") pod \"cluster-samples-operator-56fcf86bf-dmclb\" (UID: \"165ea525-4d56-41f2-85b8-772b6abe82ff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"
Jul 09 12:03:13.161846 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161774    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3b9d000a-c712-4679-8044-c0a4f18920f0-snapshots\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.161846 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.161783    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 12:03:13.161846 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161802    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/24691895-ba59-41b7-8cb0-eb97b4708875-config\") pod \"service-ca-operator-5d46d9ffd4-8g6sz\" (UID: \"24691895-ba59-41b7-8cb0-eb97b4708875\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz"
Jul 09 12:03:13.161846 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161827    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plr42\" (UniqueName: \"kubernetes.io/projected/a7de5580-08d4-449f-9f76-7fab78680c74-kube-api-access-plr42\") pod \"network-check-source-989f8b8dc-zjkp8\" (UID: \"a7de5580-08d4-449f-9f76-7fab78680c74\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zjkp8"
Jul 09 12:03:13.161846 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.161850    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:03:13.661831739 +0000 UTC m=+66.339360775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : secret "samples-operator-tls" not found
Jul 09 12:03:13.162232 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161890    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-ca-trust-extracted\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.162232 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161914    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b9d000a-c712-4679-8044-c0a4f18920f0-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.162232 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161920    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-installation-pull-secrets\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.162232 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161949    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.162232 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.161964    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-certificates\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.162232 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.162025    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 12:03:13.162232 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.162038    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-787c4f4479-255zf: secret "image-registry-tls" not found
Jul 09 12:03:13.162232 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.162077    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls podName:87bb046d-1a4c-4eec-aa6b-812bdc1e5b21 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:13.662062041 +0000 UTC m=+66.339591080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls") pod "image-registry-787c4f4479-255zf" (UID: "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21") : secret "image-registry-tls" not found
Jul 09 12:03:13.162622 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.162452    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-trusted-ca\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.162622 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.162616    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3b9d000a-c712-4679-8044-c0a4f18920f0-snapshots\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.163001 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.162975    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b9d000a-c712-4679-8044-c0a4f18920f0-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.163164 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.162999    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-ca-trust-extracted\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.163366 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.163347    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/24691895-ba59-41b7-8cb0-eb97b4708875-config\") pod \"service-ca-operator-5d46d9ffd4-8g6sz\" (UID: \"24691895-ba59-41b7-8cb0-eb97b4708875\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz"
Jul 09 12:03:13.164387 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.164365    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3b9d000a-c712-4679-8044-c0a4f18920f0-serving-cert\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.164461 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.164443    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-image-registry-private-configuration\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.164495 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.164450    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24691895-ba59-41b7-8cb0-eb97b4708875-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-8g6sz\" (UID: \"24691895-ba59-41b7-8cb0-eb97b4708875\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz"
Jul 09 12:03:13.164650 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.164635    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-installation-pull-secrets\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.171221 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.170955    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 12:03:13.171221 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.171164    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-bound-sa-token\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.171728 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.171682    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmw4j\" (UniqueName: \"kubernetes.io/projected/24691895-ba59-41b7-8cb0-eb97b4708875-kube-api-access-lmw4j\") pod \"service-ca-operator-5d46d9ffd4-8g6sz\" (UID: \"24691895-ba59-41b7-8cb0-eb97b4708875\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz"
Jul 09 12:03:13.172380 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.172361    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46x6k\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-kube-api-access-46x6k\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.172777 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.172753    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnmwg\" (UniqueName: \"kubernetes.io/projected/d4216769-1d00-4c9b-b986-a577ad840711-kube-api-access-wnmwg\") pod \"volume-data-source-validator-5d456fdbcd-xl7np\" (UID: \"d4216769-1d00-4c9b-b986-a577ad840711\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xl7np"
Jul 09 12:03:13.173000 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.172982    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zzt8\" (UniqueName: \"kubernetes.io/projected/3b9d000a-c712-4679-8044-c0a4f18920f0-kube-api-access-6zzt8\") pod \"insights-operator-5c847dcfc4-k662b\" (UID: \"3b9d000a-c712-4679-8044-c0a4f18920f0\") " pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.173206 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.173188    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltx9j\" (UniqueName: \"kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j\") pod \"cluster-samples-operator-56fcf86bf-dmclb\" (UID: \"165ea525-4d56-41f2-85b8-772b6abe82ff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"
Jul 09 12:03:13.182224 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.182208    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 12:03:13.192526 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.192506    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plr42\" (UniqueName: \"kubernetes.io/projected/a7de5580-08d4-449f-9f76-7fab78680c74-kube-api-access-plr42\") pod \"network-check-source-989f8b8dc-zjkp8\" (UID: \"a7de5580-08d4-449f-9f76-7fab78680c74\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zjkp8"
Jul 09 12:03:13.244422 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.244398    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz"
Jul 09 12:03:13.257675 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.257658    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xl7np"
Jul 09 12:03:13.291545 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.291508    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-k662b"
Jul 09 12:03:13.341954 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.341928    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zjkp8"
Jul 09 12:03:13.404039 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.402973    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz"]
Jul 09 12:03:13.407252 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:03:13.407145    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24691895_ba59_41b7_8cb0_eb97b4708875.slice/crio-44b5c285bd50d381f27583bd00228cddbedda624a3dcfb824c97947e128d3ea1 WatchSource:0}: Error finding container 44b5c285bd50d381f27583bd00228cddbedda624a3dcfb824c97947e128d3ea1: Status 404 returned error can't find the container with id 44b5c285bd50d381f27583bd00228cddbedda624a3dcfb824c97947e128d3ea1
Jul 09 12:03:13.426350 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.426313    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xl7np"]
Jul 09 12:03:13.429488 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:03:13.429421    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4216769_1d00_4c9b_b986_a577ad840711.slice/crio-8d2d707ecb1063a2d456b6a1ff3842081b6b8cce255557c8773c447bc843887e WatchSource:0}: Error finding container 8d2d707ecb1063a2d456b6a1ff3842081b6b8cce255557c8773c447bc843887e: Status 404 returned error can't find the container with id 8d2d707ecb1063a2d456b6a1ff3842081b6b8cce255557c8773c447bc843887e
Jul 09 12:03:13.478286 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.478260    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-k662b"]
Jul 09 12:03:13.481461 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:03:13.481433    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b9d000a_c712_4679_8044_c0a4f18920f0.slice/crio-7e896f98f1fd22055b2d2fceb7f53b091a130bcc7f157b13aeab6426c1feeb6d WatchSource:0}: Error finding container 7e896f98f1fd22055b2d2fceb7f53b091a130bcc7f157b13aeab6426c1feeb6d: Status 404 returned error can't find the container with id 7e896f98f1fd22055b2d2fceb7f53b091a130bcc7f157b13aeab6426c1feeb6d
Jul 09 12:03:13.489923 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.489902    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-zjkp8"]
Jul 09 12:03:13.492763 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:03:13.492739    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7de5580_08d4_449f_9f76_7fab78680c74.slice/crio-6f2c933e0af56429a1575e71d0356ac7f5a1066842ead4ecd67cfd074a9248fe WatchSource:0}: Error finding container 6f2c933e0af56429a1575e71d0356ac7f5a1066842ead4ecd67cfd074a9248fe: Status 404 returned error can't find the container with id 6f2c933e0af56429a1575e71d0356ac7f5a1066842ead4ecd67cfd074a9248fe
Jul 09 12:03:13.564839 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.564807    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:13.564962 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.564854    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs\") pod \"network-metrics-daemon-rltxg\" (UID: \"4ac6e98e-e488-4741-aa67-2a6a130291d8\") " pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:03:13.564962 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.564915    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:03:13.565057 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.564967    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 12:03:13.565057 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.565027    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls podName:ba5787df-e1ac-4ae8-af04-901fc636f023 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:14.565009561 +0000 UTC m=+67.242538612 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xnp29" (UID: "ba5787df-e1ac-4ae8-af04-901fc636f023") : secret "cluster-monitoring-operator-tls" not found
Jul 09 12:03:13.567424 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.567408    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 12:03:13.567549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.567533    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 12:03:13.576106 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.576091    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 09 12:03:13.576149 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.576138    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs podName:4ac6e98e-e488-4741-aa67-2a6a130291d8 nodeName:}" failed. No retries permitted until 2026-07-09 12:04:17.576125037 +0000 UTC m=+130.253654072 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs") pod "network-metrics-daemon-rltxg" (UID: "4ac6e98e-e488-4741-aa67-2a6a130291d8") : secret "metrics-daemon-secret" not found
Jul 09 12:03:13.578187 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.578171    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b-original-pull-secret\") pod \"global-pull-secret-syncer-p456j\" (UID: \"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b\") " pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:03:13.665605 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.665517    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x28d\" (UniqueName: \"kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d\") pod \"network-check-target-p9wwl\" (UID: \"4afb2515-0939-4e93-9d3d-7f5662d0856e\") " pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:03:13.665605 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.665572    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dmclb\" (UID: \"165ea525-4d56-41f2-85b8-772b6abe82ff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"
Jul 09 12:03:13.665821 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.665628    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:13.665821 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.665744    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 12:03:13.665821 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.665780    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 12:03:13.665821 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.665795    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-787c4f4479-255zf: secret "image-registry-tls" not found
Jul 09 12:03:13.665821 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.665804    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:03:14.665788712 +0000 UTC m=+67.343317751 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : secret "samples-operator-tls" not found
Jul 09 12:03:13.666003 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:13.665841    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls podName:87bb046d-1a4c-4eec-aa6b-812bdc1e5b21 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:14.665826761 +0000 UTC m=+67.343355814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls") pod "image-registry-787c4f4479-255zf" (UID: "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21") : secret "image-registry-tls" not found
Jul 09 12:03:13.668046 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.668029    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x28d\" (UniqueName: \"kubernetes.io/projected/4afb2515-0939-4e93-9d3d-7f5662d0856e-kube-api-access-8x28d\") pod \"network-check-target-p9wwl\" (UID: \"4afb2515-0939-4e93-9d3d-7f5662d0856e\") " pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:03:13.702778 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.702756    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cpf42\""
Jul 09 12:03:13.710402 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.710386    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:03:13.712654 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.712639    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p456j"
Jul 09 12:03:13.838160 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.838129    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p9wwl"]
Jul 09 12:03:13.841599 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:03:13.841570    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4afb2515_0939_4e93_9d3d_7f5662d0856e.slice/crio-70eda0f567cb5fb45cbac06ec38ae229c4869eb477b68d3d52b68538e7c0f3e4 WatchSource:0}: Error finding container 70eda0f567cb5fb45cbac06ec38ae229c4869eb477b68d3d52b68538e7c0f3e4: Status 404 returned error can't find the container with id 70eda0f567cb5fb45cbac06ec38ae229c4869eb477b68d3d52b68538e7c0f3e4
Jul 09 12:03:13.855141 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:13.855116    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p456j"]
Jul 09 12:03:13.857386 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:03:13.857362    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3382b7a_7abc_4a2c_b06c_c1e8f7d1310b.slice/crio-a49e96d87720c3b6f1b590298bce37773e62c5b1ca93d4951aa20a86f6ef54df WatchSource:0}: Error finding container a49e96d87720c3b6f1b590298bce37773e62c5b1ca93d4951aa20a86f6ef54df: Status 404 returned error can't find the container with id a49e96d87720c3b6f1b590298bce37773e62c5b1ca93d4951aa20a86f6ef54df
Jul 09 12:03:14.199072 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:14.199000    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p456j" event={"ID":"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b","Type":"ContainerStarted","Data":"a49e96d87720c3b6f1b590298bce37773e62c5b1ca93d4951aa20a86f6ef54df"}
Jul 09 12:03:14.201957 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:14.201924    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-k662b" event={"ID":"3b9d000a-c712-4679-8044-c0a4f18920f0","Type":"ContainerStarted","Data":"7e896f98f1fd22055b2d2fceb7f53b091a130bcc7f157b13aeab6426c1feeb6d"}
Jul 09 12:03:14.205479 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:14.205444    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xl7np" event={"ID":"d4216769-1d00-4c9b-b986-a577ad840711","Type":"ContainerStarted","Data":"8d2d707ecb1063a2d456b6a1ff3842081b6b8cce255557c8773c447bc843887e"}
Jul 09 12:03:14.206958 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:14.206900    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zjkp8" event={"ID":"a7de5580-08d4-449f-9f76-7fab78680c74","Type":"ContainerStarted","Data":"6f2c933e0af56429a1575e71d0356ac7f5a1066842ead4ecd67cfd074a9248fe"}
Jul 09 12:03:14.208159 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:14.208136    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz" event={"ID":"24691895-ba59-41b7-8cb0-eb97b4708875","Type":"ContainerStarted","Data":"44b5c285bd50d381f27583bd00228cddbedda624a3dcfb824c97947e128d3ea1"}
Jul 09 12:03:14.209366 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:14.209347    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p9wwl" event={"ID":"4afb2515-0939-4e93-9d3d-7f5662d0856e","Type":"ContainerStarted","Data":"70eda0f567cb5fb45cbac06ec38ae229c4869eb477b68d3d52b68538e7c0f3e4"}
Jul 09 12:03:14.574991 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:14.574950    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:14.575194 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:14.575104    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 12:03:14.575605 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:14.575588    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls podName:ba5787df-e1ac-4ae8-af04-901fc636f023 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:16.575563751 +0000 UTC m=+69.253092792 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xnp29" (UID: "ba5787df-e1ac-4ae8-af04-901fc636f023") : secret "cluster-monitoring-operator-tls" not found
Jul 09 12:03:14.676765 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:14.675877    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dmclb\" (UID: \"165ea525-4d56-41f2-85b8-772b6abe82ff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"
Jul 09 12:03:14.676765 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:14.675954    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:14.676765 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:14.676135    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 12:03:14.676765 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:14.676153    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-787c4f4479-255zf: secret "image-registry-tls" not found
Jul 09 12:03:14.676765 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:14.676214    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls podName:87bb046d-1a4c-4eec-aa6b-812bdc1e5b21 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:16.676194311 +0000 UTC m=+69.353723361 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls") pod "image-registry-787c4f4479-255zf" (UID: "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21") : secret "image-registry-tls" not found
Jul 09 12:03:14.676765 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:14.676619    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 12:03:14.676765 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:14.676674    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:03:16.676656662 +0000 UTC m=+69.354185701 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : secret "samples-operator-tls" not found
Jul 09 12:03:14.979030 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:14.978394    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:03:14.979030 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:14.978545    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 12:03:14.979030 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:14.978605    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls podName:f77b9023-6512-43a6-8d89-2ea512bc8c10 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:46.978587272 +0000 UTC m=+99.656116314 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls") pod "dns-default-xw2dj" (UID: "f77b9023-6512-43a6-8d89-2ea512bc8c10") : secret "dns-default-metrics-tls" not found
Jul 09 12:03:16.219565 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:16.219518    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xl7np" event={"ID":"d4216769-1d00-4c9b-b986-a577ad840711","Type":"ContainerStarted","Data":"6db5675623914c56b61464d4f51f047eda53583b3a8bf64c9348fbaa86d1dea5"}
Jul 09 12:03:16.238932 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:16.238520    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-xl7np" podStartSLOduration=2.149451819 podStartE2EDuration="4.238504499s" totalImagesPullingTime="2.08905268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:03:16.237384128 +0000 UTC m=+68.914913189" watchObservedRunningTime="2026-07-09 12:03:16.238504499 +0000 UTC m=+68.916033556"
Jul 09 12:03:16.593139 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:16.593095    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:16.593367 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:16.593343    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 12:03:16.593493 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:16.593416    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls podName:ba5787df-e1ac-4ae8-af04-901fc636f023 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:20.593396653 +0000 UTC m=+73.270925702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xnp29" (UID: "ba5787df-e1ac-4ae8-af04-901fc636f023") : secret "cluster-monitoring-operator-tls" not found
Jul 09 12:03:16.693721 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:16.693663    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:16.693901 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:16.693795    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dmclb\" (UID: \"165ea525-4d56-41f2-85b8-772b6abe82ff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"
Jul 09 12:03:16.693901 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:16.693838    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 12:03:16.693901 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:16.693858    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-787c4f4479-255zf: secret "image-registry-tls" not found
Jul 09 12:03:16.694061 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:16.693901    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 12:03:16.694061 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:16.693960    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls podName:87bb046d-1a4c-4eec-aa6b-812bdc1e5b21 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:20.693941775 +0000 UTC m=+73.371470814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls") pod "image-registry-787c4f4479-255zf" (UID: "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21") : secret "image-registry-tls" not found
Jul 09 12:03:16.694061 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:16.693987    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:03:20.693967109 +0000 UTC m=+73.371496147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : secret "samples-operator-tls" not found
Jul 09 12:03:20.627018 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:20.626976    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:20.627490 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:20.627144    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 12:03:20.627490 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:20.627228    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls podName:ba5787df-e1ac-4ae8-af04-901fc636f023 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:28.627208032 +0000 UTC m=+81.304737069 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xnp29" (UID: "ba5787df-e1ac-4ae8-af04-901fc636f023") : secret "cluster-monitoring-operator-tls" not found
Jul 09 12:03:20.728249 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:20.728216    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:20.728409 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:20.728327    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dmclb\" (UID: \"165ea525-4d56-41f2-85b8-772b6abe82ff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"
Jul 09 12:03:20.728409 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:20.728384    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 12:03:20.728409 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:20.728407    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-787c4f4479-255zf: secret "image-registry-tls" not found
Jul 09 12:03:20.728526 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:20.728464    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 12:03:20.728526 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:20.728473    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls podName:87bb046d-1a4c-4eec-aa6b-812bdc1e5b21 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:28.728453468 +0000 UTC m=+81.405982506 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls") pod "image-registry-787c4f4479-255zf" (UID: "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21") : secret "image-registry-tls" not found
Jul 09 12:03:20.728526 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:20.728513    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:03:28.728498324 +0000 UTC m=+81.406027375 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : secret "samples-operator-tls" not found
Jul 09 12:03:21.240408 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.240371    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p456j" event={"ID":"e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b","Type":"ContainerStarted","Data":"e170fe531a568f478bfb1cd9078f19c753d85c675e13b828be7d8494e1d66095"}
Jul 09 12:03:21.242085 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.242055    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-k662b" event={"ID":"3b9d000a-c712-4679-8044-c0a4f18920f0","Type":"ContainerStarted","Data":"91af97012f80d41c420a0cf14a13e9e57a6cf972404e960bba71f9bbd220843a"}
Jul 09 12:03:21.243648 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.243615    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zjkp8" event={"ID":"a7de5580-08d4-449f-9f76-7fab78680c74","Type":"ContainerStarted","Data":"de1982b5953d89eede25b40ea53ac27734de3f10ce7e1426bd363950d083223f"}
Jul 09 12:03:21.245268 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.245246    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz" event={"ID":"24691895-ba59-41b7-8cb0-eb97b4708875","Type":"ContainerStarted","Data":"70542542c7430630c424cf96ec46050e0bb63e3456f713f13efcff0861378584"}
Jul 09 12:03:21.246807 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.246785    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p9wwl" event={"ID":"4afb2515-0939-4e93-9d3d-7f5662d0856e","Type":"ContainerStarted","Data":"29a6797cc95627fc56a5517085a0dfb3f1371f9c8f49fed4bef7ec406b05e883"}
Jul 09 12:03:21.246948 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.246930    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:03:21.257167 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.257122    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p456j" podStartSLOduration=67.022734856 podStartE2EDuration="1m13.25710503s" totalImagesPullingTime="6.234370174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:03:21.256050032 +0000 UTC m=+73.933579103" watchObservedRunningTime="2026-07-09 12:03:21.25710503 +0000 UTC m=+73.934634085"
Jul 09 12:03:21.275743 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.275680    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz" podStartSLOduration=2.592104426 podStartE2EDuration="9.275661643s" totalImagesPullingTime="6.683557217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:03:21.274803335 +0000 UTC m=+73.952332391" watchObservedRunningTime="2026-07-09 12:03:21.275661643 +0000 UTC m=+73.953190703"
Jul 09 12:03:21.294775 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.294722    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-k662b" podStartSLOduration=2.684984714 podStartE2EDuration="9.294687931s" totalImagesPullingTime="6.609703217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:03:21.293835534 +0000 UTC m=+73.971364592" watchObservedRunningTime="2026-07-09 12:03:21.294687931 +0000 UTC m=+73.972216991"
Jul 09 12:03:21.314178 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.314080    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zjkp8" podStartSLOduration=1.7149373730000002 podStartE2EDuration="8.314062025s" totalImagesPullingTime="6.599124652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:03:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:03:21.313452874 +0000 UTC m=+73.990981933" watchObservedRunningTime="2026-07-09 12:03:21.314062025 +0000 UTC m=+73.991591085"
Jul 09 12:03:21.331474 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.331271    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-p9wwl" podStartSLOduration=67.070727062 podStartE2EDuration="1m13.331254756s" totalImagesPullingTime="6.260527694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:03:21.330738574 +0000 UTC m=+74.008267626" watchObservedRunningTime="2026-07-09 12:03:21.331254756 +0000 UTC m=+74.008783806"
Jul 09 12:03:21.781962 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.778749    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8rrb2"]
Jul 09 12:03:21.786603 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.786577    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:21.789089 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.789064    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 12:03:21.789246 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.789065    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 12:03:21.789540 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.789523    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fvw8k\""
Jul 09 12:03:21.797582 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.797560    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8rrb2"]
Jul 09 12:03:21.838678 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.838649    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b9c3ac6-03da-448e-b885-4e7edee28684-crio-socket\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:21.838678 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.838683    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:21.838936 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.838796    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b9c3ac6-03da-448e-b885-4e7edee28684-data-volume\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:21.838936 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.838866    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:21.838936 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.838898    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vswl\" (UniqueName: \"kubernetes.io/projected/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-api-access-6vswl\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:21.939450 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.939416    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b9c3ac6-03da-448e-b885-4e7edee28684-crio-socket\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:21.939450 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.939452    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:21.939671 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.939496    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b9c3ac6-03da-448e-b885-4e7edee28684-data-volume\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:21.939671 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.939544    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:21.939671 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.939573    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vswl\" (UniqueName: \"kubernetes.io/projected/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-api-access-6vswl\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:21.939671 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.939542    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b9c3ac6-03da-448e-b885-4e7edee28684-crio-socket\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:21.939671 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:21.939575    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 12:03:21.939671 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:21.939637    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 12:03:21.939671 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:21.939659    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls podName:2b9c3ac6-03da-448e-b885-4e7edee28684 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:22.439639796 +0000 UTC m=+75.117168835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8rrb2" (UID: "2b9c3ac6-03da-448e-b885-4e7edee28684") : secret "insights-runtime-extractor-tls" not found
Jul 09 12:03:21.939975 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:21.939729    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm podName:2b9c3ac6-03da-448e-b885-4e7edee28684 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:22.439687347 +0000 UTC m=+75.117216406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8rrb2" (UID: "2b9c3ac6-03da-448e-b885-4e7edee28684") : configmap "kube-rbac-proxy" not found
Jul 09 12:03:21.939975 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.939857    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b9c3ac6-03da-448e-b885-4e7edee28684-data-volume\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:21.951685 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:21.951640    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vswl\" (UniqueName: \"kubernetes.io/projected/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-api-access-6vswl\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:22.443838 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:22.443804    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:22.443998 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:22.443877    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:22.443998 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:22.443924    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 12:03:22.443998 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:22.443968    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 12:03:22.443998 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:22.443997    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm podName:2b9c3ac6-03da-448e-b885-4e7edee28684 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:23.443979986 +0000 UTC m=+76.121509022 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8rrb2" (UID: "2b9c3ac6-03da-448e-b885-4e7edee28684") : configmap "kube-rbac-proxy" not found
Jul 09 12:03:22.444124 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:22.444016    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls podName:2b9c3ac6-03da-448e-b885-4e7edee28684 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:23.444004661 +0000 UTC m=+76.121533696 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8rrb2" (UID: "2b9c3ac6-03da-448e-b885-4e7edee28684") : secret "insights-runtime-extractor-tls" not found
Jul 09 12:03:23.451820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:23.451775    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:23.452216 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:23.451858    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:23.452216 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:23.451922    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 12:03:23.452216 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:23.451935    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 12:03:23.452216 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:23.451985    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls podName:2b9c3ac6-03da-448e-b885-4e7edee28684 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:25.451970176 +0000 UTC m=+78.129499212 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8rrb2" (UID: "2b9c3ac6-03da-448e-b885-4e7edee28684") : secret "insights-runtime-extractor-tls" not found
Jul 09 12:03:23.452216 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:23.452001    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm podName:2b9c3ac6-03da-448e-b885-4e7edee28684 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:25.451994911 +0000 UTC m=+78.129523946 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8rrb2" (UID: "2b9c3ac6-03da-448e-b885-4e7edee28684") : configmap "kube-rbac-proxy" not found
Jul 09 12:03:24.850800 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:24.850770    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zjcb9_760d11e4-c839-41a2-8e35-b4d2566ef960/dns-node-resolver/0.log"
Jul 09 12:03:25.468126 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:25.468091    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:25.468287 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:25.468157    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:25.468287 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:25.468231    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 12:03:25.468287 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:25.468254    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 12:03:25.468389 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:25.468291    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm podName:2b9c3ac6-03da-448e-b885-4e7edee28684 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:29.468275989 +0000 UTC m=+82.145805027 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8rrb2" (UID: "2b9c3ac6-03da-448e-b885-4e7edee28684") : configmap "kube-rbac-proxy" not found
Jul 09 12:03:25.468389 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:25.468310    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls podName:2b9c3ac6-03da-448e-b885-4e7edee28684 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:29.468297005 +0000 UTC m=+82.145826041 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8rrb2" (UID: "2b9c3ac6-03da-448e-b885-4e7edee28684") : secret "insights-runtime-extractor-tls" not found
Jul 09 12:03:25.850553 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:25.850521    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r5k4m_8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2/node-ca/0.log"
Jul 09 12:03:26.250728 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:26.250621    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dcmlh_f68292df-cc6d-4528-bac8-3a6badb20ed2/serve-healthcheck-canary/0.log"
Jul 09 12:03:28.693195 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:28.693158    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:28.693568 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:28.693308    2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 12:03:28.693568 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:28.693376    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls podName:ba5787df-e1ac-4ae8-af04-901fc636f023 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:44.693359506 +0000 UTC m=+97.370888562 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xnp29" (UID: "ba5787df-e1ac-4ae8-af04-901fc636f023") : secret "cluster-monitoring-operator-tls" not found
Jul 09 12:03:28.794237 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:28.794136    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dmclb\" (UID: \"165ea525-4d56-41f2-85b8-772b6abe82ff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"
Jul 09 12:03:28.794237 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:28.794189    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:28.794403 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:28.794281    2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 12:03:28.794403 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:28.794283    2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 12:03:28.794403 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:28.794292    2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-787c4f4479-255zf: secret "image-registry-tls" not found
Jul 09 12:03:28.794403 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:28.794346    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls podName:87bb046d-1a4c-4eec-aa6b-812bdc1e5b21 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:44.794329353 +0000 UTC m=+97.471858409 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls") pod "image-registry-787c4f4479-255zf" (UID: "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21") : secret "image-registry-tls" not found
Jul 09 12:03:28.794403 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:28.794359    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:03:44.794352701 +0000 UTC m=+97.471881736 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : secret "samples-operator-tls" not found
Jul 09 12:03:29.497465 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:29.497416    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:29.497654 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:29.497512    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:29.497654 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:29.497553    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 12:03:29.497654 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:29.497618    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 12:03:29.497654 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:29.497631    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm podName:2b9c3ac6-03da-448e-b885-4e7edee28684 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:37.497613737 +0000 UTC m=+90.175142793 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8rrb2" (UID: "2b9c3ac6-03da-448e-b885-4e7edee28684") : configmap "kube-rbac-proxy" not found
Jul 09 12:03:29.497654 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:29.497656    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls podName:2b9c3ac6-03da-448e-b885-4e7edee28684 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:37.497645226 +0000 UTC m=+90.175174262 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8rrb2" (UID: "2b9c3ac6-03da-448e-b885-4e7edee28684") : secret "insights-runtime-extractor-tls" not found
Jul 09 12:03:37.558337 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:37.558301    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:37.558732 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:37.558400    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:37.558732 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:37.558524    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 12:03:37.558732 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:03:37.558594    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls podName:2b9c3ac6-03da-448e-b885-4e7edee28684 nodeName:}" failed. No retries permitted until 2026-07-09 12:03:53.558574439 +0000 UTC m=+106.236103477 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8rrb2" (UID: "2b9c3ac6-03da-448e-b885-4e7edee28684") : secret "insights-runtime-extractor-tls" not found
Jul 09 12:03:37.558973 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:37.558950    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b9c3ac6-03da-448e-b885-4e7edee28684-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:44.714287 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:44.714245    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:44.716813 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:44.716784    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ba5787df-e1ac-4ae8-af04-901fc636f023-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xnp29\" (UID: \"ba5787df-e1ac-4ae8-af04-901fc636f023\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:44.815188 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:44.815141    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:44.815338 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:44.815253    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dmclb\" (UID: \"165ea525-4d56-41f2-85b8-772b6abe82ff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"
Jul 09 12:03:44.817630 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:44.817602    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls\") pod \"image-registry-787c4f4479-255zf\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") " pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:44.817630 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:44.817616    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/165ea525-4d56-41f2-85b8-772b6abe82ff-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dmclb\" (UID: \"165ea525-4d56-41f2-85b8-772b6abe82ff\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"
Jul 09 12:03:44.865223 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:44.865199    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:44.947161 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:44.947128    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"
Jul 09 12:03:44.994425 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:44.994345    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-787c4f4479-255zf"]
Jul 09 12:03:44.998492 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:03:44.998466    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87bb046d_1a4c_4eec_aa6b_812bdc1e5b21.slice/crio-5483459b8ad94843a35b77217f237ed1b5242c964b1d55c002681b5d7d011620 WatchSource:0}: Error finding container 5483459b8ad94843a35b77217f237ed1b5242c964b1d55c002681b5d7d011620: Status 404 returned error can't find the container with id 5483459b8ad94843a35b77217f237ed1b5242c964b1d55c002681b5d7d011620
Jul 09 12:03:45.071718 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:45.071675    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"
Jul 09 12:03:45.078796 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:45.078696    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29"]
Jul 09 12:03:45.081280 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:03:45.081250    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba5787df_e1ac_4ae8_af04_901fc636f023.slice/crio-55e45675a51fd9d57b861faa56d8d8507be4a5f2bb57664f18b7af9c924b11ad WatchSource:0}: Error finding container 55e45675a51fd9d57b861faa56d8d8507be4a5f2bb57664f18b7af9c924b11ad: Status 404 returned error can't find the container with id 55e45675a51fd9d57b861faa56d8d8507be4a5f2bb57664f18b7af9c924b11ad
Jul 09 12:03:45.197430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:45.197395    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb"]
Jul 09 12:03:45.321962 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:45.321924    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerStarted","Data":"f0300e8a19a57eea00a1dc437bea5e44ed95e094d18dac06c922bc0c6ea2868f"}
Jul 09 12:03:45.322136 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:45.321991    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerStarted","Data":"5483459b8ad94843a35b77217f237ed1b5242c964b1d55c002681b5d7d011620"}
Jul 09 12:03:45.322136 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:45.322049    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:03:45.323205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:45.323179    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb" event={"ID":"165ea525-4d56-41f2-85b8-772b6abe82ff","Type":"ContainerStarted","Data":"6d3305a4acd0ae4f28dfa2805e5845afb1287a67551c357ff64408bb3f102100"}
Jul 09 12:03:45.324349 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:45.324330    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29" event={"ID":"ba5787df-e1ac-4ae8-af04-901fc636f023","Type":"ContainerStarted","Data":"55e45675a51fd9d57b861faa56d8d8507be4a5f2bb57664f18b7af9c924b11ad"}
Jul 09 12:03:45.342615 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:45.342565    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podStartSLOduration=32.342549277 podStartE2EDuration="32.342549277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:03:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:03:45.341235945 +0000 UTC m=+98.018765003" watchObservedRunningTime="2026-07-09 12:03:45.342549277 +0000 UTC m=+98.020078336"
Jul 09 12:03:47.033617 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:47.033469    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:03:47.036304 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:47.036282    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f77b9023-6512-43a6-8d89-2ea512bc8c10-metrics-tls\") pod \"dns-default-xw2dj\" (UID: \"f77b9023-6512-43a6-8d89-2ea512bc8c10\") " pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:03:47.093334 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:47.093298    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mzdcd\""
Jul 09 12:03:47.100822 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:47.100795    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:03:48.287170 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:48.287142    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xw2dj"]
Jul 09 12:03:48.290125 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:03:48.290093    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf77b9023_6512_43a6_8d89_2ea512bc8c10.slice/crio-de4064bf80a35a30aa6196966c419a0eba21b7a87f034ccfddeddada9265c3e3 WatchSource:0}: Error finding container de4064bf80a35a30aa6196966c419a0eba21b7a87f034ccfddeddada9265c3e3: Status 404 returned error can't find the container with id de4064bf80a35a30aa6196966c419a0eba21b7a87f034ccfddeddada9265c3e3
Jul 09 12:03:48.336933 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:48.336896    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb" event={"ID":"165ea525-4d56-41f2-85b8-772b6abe82ff","Type":"ContainerStarted","Data":"7878277150d57915007af4b2b479d5b6180276fbd1a6d4219b62c6166f50bc11"}
Jul 09 12:03:48.338600 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:48.338573    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29" event={"ID":"ba5787df-e1ac-4ae8-af04-901fc636f023","Type":"ContainerStarted","Data":"fbbba3f98d1e12baf53f701bd7d3784b48e0b2c59a4fce3e916ae5330d843b18"}
Jul 09 12:03:48.339774 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:48.339749    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xw2dj" event={"ID":"f77b9023-6512-43a6-8d89-2ea512bc8c10","Type":"ContainerStarted","Data":"de4064bf80a35a30aa6196966c419a0eba21b7a87f034ccfddeddada9265c3e3"}
Jul 09 12:03:48.360333 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:48.360274    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xnp29" podStartSLOduration=33.287348206 podStartE2EDuration="36.360252406s" totalImagesPullingTime="3.0729042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:03:48.359647587 +0000 UTC m=+101.037176647" watchObservedRunningTime="2026-07-09 12:03:48.360252406 +0000 UTC m=+101.037781465"
Jul 09 12:03:49.345141 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:49.345105    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb" event={"ID":"165ea525-4d56-41f2-85b8-772b6abe82ff","Type":"ContainerStarted","Data":"7571c297d76876bfef7147f8085e57f9188d21cb0ecc407fa787a9ac4ab1f0da"}
Jul 09 12:03:49.363163 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:49.363100    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb" podStartSLOduration=34.454600211 podStartE2EDuration="37.363078421s" totalImagesPullingTime="2.90847821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:03:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:03:49.360974849 +0000 UTC m=+102.038503908" watchObservedRunningTime="2026-07-09 12:03:49.363078421 +0000 UTC m=+102.040607480"
Jul 09 12:03:51.353572 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:51.353538    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xw2dj" event={"ID":"f77b9023-6512-43a6-8d89-2ea512bc8c10","Type":"ContainerStarted","Data":"f1c7a54c60e7265269ddf0c25e00b7f04009c5774f21d21d0302ea0078f3e9f1"}
Jul 09 12:03:51.354066 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:51.353595    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xw2dj" event={"ID":"f77b9023-6512-43a6-8d89-2ea512bc8c10","Type":"ContainerStarted","Data":"578b93b8cd86e05091ad0289e3372e6aaacc8ea2a397d94a179f41fcc0f3292f"}
Jul 09 12:03:51.354066 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:51.353657    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:03:51.371645 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:51.371604    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xw2dj" podStartSLOduration=66.022380715 podStartE2EDuration="1m8.371569376s" totalImagesPullingTime="2.349188661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:03:51.37102142 +0000 UTC m=+104.048550477" watchObservedRunningTime="2026-07-09 12:03:51.371569376 +0000 UTC m=+104.049098435"
Jul 09 12:03:52.253834 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:52.253804    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-p9wwl"
Jul 09 12:03:53.588797 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:53.588756    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:53.591247 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:53.591211    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b9c3ac6-03da-448e-b885-4e7edee28684-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8rrb2\" (UID: \"2b9c3ac6-03da-448e-b885-4e7edee28684\") " pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:53.596207 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:53.596184    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8rrb2"
Jul 09 12:03:53.724786 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:53.724635    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8rrb2"]
Jul 09 12:03:53.727348 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:03:53.727310    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b9c3ac6_03da_448e_b885_4e7edee28684.slice/crio-61ccc3b7fbbca07be7eea0f3072e866bc042af387c1a5b0252b2c3d43385f79d WatchSource:0}: Error finding container 61ccc3b7fbbca07be7eea0f3072e866bc042af387c1a5b0252b2c3d43385f79d: Status 404 returned error can't find the container with id 61ccc3b7fbbca07be7eea0f3072e866bc042af387c1a5b0252b2c3d43385f79d
Jul 09 12:03:54.368968 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:54.368931    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8rrb2" event={"ID":"2b9c3ac6-03da-448e-b885-4e7edee28684","Type":"ContainerStarted","Data":"61ccc3b7fbbca07be7eea0f3072e866bc042af387c1a5b0252b2c3d43385f79d"}
Jul 09 12:03:58.385940 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:58.385909    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8rrb2" event={"ID":"2b9c3ac6-03da-448e-b885-4e7edee28684","Type":"ContainerStarted","Data":"f1acc1ef1ca13f4e46197b7530c9a7938d5462086f33848ed3ec874435e76b6e"}
Jul 09 12:03:58.386298 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:03:58.385964    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8rrb2" event={"ID":"2b9c3ac6-03da-448e-b885-4e7edee28684","Type":"ContainerStarted","Data":"aa60735283594d52c0638a8ee3b2d9c5bcf3d134e3d6f61e352899c0343c58b9"}
Jul 09 12:04:01.363171 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:01.363142    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xw2dj"
Jul 09 12:04:01.399056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:01.399020    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8rrb2" event={"ID":"2b9c3ac6-03da-448e-b885-4e7edee28684","Type":"ContainerStarted","Data":"da49c927058eaaebc7c2ba8b8708d3b85bd35f07a1050937f4cddb240dcda5a7"}
Jul 09 12:04:01.419369 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:01.419325    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8rrb2" podStartSLOduration=33.51393364 podStartE2EDuration="40.419309535s" totalImagesPullingTime="6.905375895s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:03:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:04:01.418921359 +0000 UTC m=+114.096450427" watchObservedRunningTime="2026-07-09 12:04:01.419309535 +0000 UTC m=+114.096838593"
Jul 09 12:04:04.870037 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:04.869997    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:04:04.870467 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:04.870053    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:04:06.332684 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:06.332645    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:04:06.333069 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:06.332694    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:04:14.869309 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:14.869275    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:04:14.869725 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:14.869326    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:04:16.333116 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:16.333085    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:04:16.333482 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:16.333142    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:04:17.666328 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:17.666281    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs\") pod \"network-metrics-daemon-rltxg\" (UID: \"4ac6e98e-e488-4741-aa67-2a6a130291d8\") " pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:04:17.668793 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:17.668768    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac6e98e-e488-4741-aa67-2a6a130291d8-metrics-certs\") pod \"network-metrics-daemon-rltxg\" (UID: \"4ac6e98e-e488-4741-aa67-2a6a130291d8\") " pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:04:17.909568 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:17.909539    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-62g6t\""
Jul 09 12:04:17.917792 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:17.917743    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rltxg"
Jul 09 12:04:18.046862 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:18.046830    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rltxg"]
Jul 09 12:04:18.049768 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:04:18.049736    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ac6e98e_e488_4741_aa67_2a6a130291d8.slice/crio-9cc0c8c4b579e19ae0fb07f45d07bc6fea0aa0b2ddd5aa7354fb6f7985c46c93 WatchSource:0}: Error finding container 9cc0c8c4b579e19ae0fb07f45d07bc6fea0aa0b2ddd5aa7354fb6f7985c46c93: Status 404 returned error can't find the container with id 9cc0c8c4b579e19ae0fb07f45d07bc6fea0aa0b2ddd5aa7354fb6f7985c46c93
Jul 09 12:04:18.456105 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:18.456072    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rltxg" event={"ID":"4ac6e98e-e488-4741-aa67-2a6a130291d8","Type":"ContainerStarted","Data":"9cc0c8c4b579e19ae0fb07f45d07bc6fea0aa0b2ddd5aa7354fb6f7985c46c93"}
Jul 09 12:04:19.461483 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:19.461404    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rltxg" event={"ID":"4ac6e98e-e488-4741-aa67-2a6a130291d8","Type":"ContainerStarted","Data":"404e13ec4b3881ed439cedfe2c6f857aada5a7494b4ab9b8ca7d913622d08d55"}
Jul 09 12:04:19.461483 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:19.461457    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rltxg" event={"ID":"4ac6e98e-e488-4741-aa67-2a6a130291d8","Type":"ContainerStarted","Data":"326869206a17dfbb17734f03d758ff19c0a95f5b4875e699a356084cefe40f0b"}
Jul 09 12:04:19.479533 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:19.479488    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rltxg" podStartSLOduration=130.415674789 podStartE2EDuration="2m11.479472952s" totalImagesPullingTime="1.063798163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:04:19.478456881 +0000 UTC m=+132.155985938" watchObservedRunningTime="2026-07-09 12:04:19.479472952 +0000 UTC m=+132.157002010"
Jul 09 12:04:24.869152 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:24.869116    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:04:24.869584 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:24.869168    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:04:24.869584 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:24.869207    2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:04:24.869679 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:24.869641    2565 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"f0300e8a19a57eea00a1dc437bea5e44ed95e094d18dac06c922bc0c6ea2868f"} pod="openshift-image-registry/image-registry-787c4f4479-255zf" containerMessage="Container registry failed liveness probe, will be restarted"
Jul 09 12:04:24.873102 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:24.873080    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:04:24.873216 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:24.873118    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:04:31.501842 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:31.501810    2565 generic.go:356] "Generic (PLEG): container finished" podID="3b9d000a-c712-4679-8044-c0a4f18920f0" containerID="91af97012f80d41c420a0cf14a13e9e57a6cf972404e960bba71f9bbd220843a" exitCode=0
Jul 09 12:04:31.502223 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:31.501880    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-k662b" event={"ID":"3b9d000a-c712-4679-8044-c0a4f18920f0","Type":"ContainerDied","Data":"91af97012f80d41c420a0cf14a13e9e57a6cf972404e960bba71f9bbd220843a"}
Jul 09 12:04:31.502223 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:31.502151    2565 scope.go:117] "RemoveContainer" containerID="91af97012f80d41c420a0cf14a13e9e57a6cf972404e960bba71f9bbd220843a"
Jul 09 12:04:32.507101 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:32.507067    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-k662b" event={"ID":"3b9d000a-c712-4679-8044-c0a4f18920f0","Type":"ContainerStarted","Data":"5ae81778a22579440c7069ccc0d1e62496d59c31fe65439bbf1d0f19848854ea"}
Jul 09 12:04:33.470821 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:33.470793    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xw2dj_f77b9023-6512-43a6-8d89-2ea512bc8c10/dns/0.log"
Jul 09 12:04:33.670585 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:33.670553    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xw2dj_f77b9023-6512-43a6-8d89-2ea512bc8c10/kube-rbac-proxy/0.log"
Jul 09 12:04:34.271025 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:34.270997    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zjcb9_760d11e4-c839-41a2-8e35-b4d2566ef960/dns-node-resolver/0.log"
Jul 09 12:04:34.671555 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:34.671525    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-787c4f4479-255zf_87bb046d-1a4c-4eec-aa6b-812bdc1e5b21/registry/0.log"
Jul 09 12:04:34.873407 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:34.873375    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:04:34.873559 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:34.873425    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:04:35.270827 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:35.270799    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r5k4m_8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2/node-ca/0.log"
Jul 09 12:04:35.675200 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:35.675171    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dcmlh_f68292df-cc6d-4528-bac8-3a6badb20ed2/serve-healthcheck-canary/0.log"
Jul 09 12:04:36.522519 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:36.522484    2565 generic.go:356] "Generic (PLEG): container finished" podID="24691895-ba59-41b7-8cb0-eb97b4708875" containerID="70542542c7430630c424cf96ec46050e0bb63e3456f713f13efcff0861378584" exitCode=0
Jul 09 12:04:36.522686 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:36.522523    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz" event={"ID":"24691895-ba59-41b7-8cb0-eb97b4708875","Type":"ContainerDied","Data":"70542542c7430630c424cf96ec46050e0bb63e3456f713f13efcff0861378584"}
Jul 09 12:04:36.522875 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:36.522859    2565 scope.go:117] "RemoveContainer" containerID="70542542c7430630c424cf96ec46050e0bb63e3456f713f13efcff0861378584"
Jul 09 12:04:37.527171 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:37.527140    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-8g6sz" event={"ID":"24691895-ba59-41b7-8cb0-eb97b4708875","Type":"ContainerStarted","Data":"15c3a6d90755908d5b538981a594960e30af8be1909ed54f3954f2a25484cab5"}
Jul 09 12:04:44.873898 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:44.873843    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:04:44.874363 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:44.873921    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:04:49.889082 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:49.889029    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" containerID="cri-o://f0300e8a19a57eea00a1dc437bea5e44ed95e094d18dac06c922bc0c6ea2868f" gracePeriod=30
Jul 09 12:04:50.571463 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:50.571430    2565 generic.go:356] "Generic (PLEG): container finished" podID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerID="f0300e8a19a57eea00a1dc437bea5e44ed95e094d18dac06c922bc0c6ea2868f" exitCode=0
Jul 09 12:04:50.571629 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:50.571504    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerDied","Data":"f0300e8a19a57eea00a1dc437bea5e44ed95e094d18dac06c922bc0c6ea2868f"}
Jul 09 12:04:50.571629 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:50.571561    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerStarted","Data":"6095e4afcab07de559504de336c73ea18399e7aba08d45808fa9058e3f71a7f5"}
Jul 09 12:04:50.571741 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:04:50.571730    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:05:04.870782 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:04.870688    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:05:04.870782 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:04.870766    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:05:11.581359 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:11.581326    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:05:11.581772 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:11.581399    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:05:14.868987 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:14.868948    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:05:14.869342 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:14.869010    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:05:21.579838 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:21.579798    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:05:21.580366 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:21.579861    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:05:24.869483 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:24.869448    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:05:24.869909 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:24.869497    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:05:24.869909 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:24.869537    2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:05:24.869997 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:24.869954    2565 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"6095e4afcab07de559504de336c73ea18399e7aba08d45808fa9058e3f71a7f5"} pod="openshift-image-registry/image-registry-787c4f4479-255zf" containerMessage="Container registry failed liveness probe, will be restarted"
Jul 09 12:05:24.873576 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:24.873549    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:05:24.873690 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:24.873598    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:05:34.874435 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:34.874404    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:05:34.874809 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:34.874453    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:05:44.874342 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:44.874305    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:05:44.874726 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:44.874369    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:05:49.889288 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:49.889250    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" containerID="cri-o://6095e4afcab07de559504de336c73ea18399e7aba08d45808fa9058e3f71a7f5" gracePeriod=30
Jul 09 12:05:50.768129 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:50.768097    2565 generic.go:356] "Generic (PLEG): container finished" podID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerID="6095e4afcab07de559504de336c73ea18399e7aba08d45808fa9058e3f71a7f5" exitCode=0
Jul 09 12:05:50.768339 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:50.768174    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerDied","Data":"6095e4afcab07de559504de336c73ea18399e7aba08d45808fa9058e3f71a7f5"}
Jul 09 12:05:50.768339 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:50.768234    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerStarted","Data":"80ca4be251b3dab821b5db148af83b104c6aa0d7beaa58fb74e222b5fde38a68"}
Jul 09 12:05:50.768339 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:50.768262    2565 scope.go:117] "RemoveContainer" containerID="f0300e8a19a57eea00a1dc437bea5e44ed95e094d18dac06c922bc0c6ea2868f"
Jul 09 12:05:50.768515 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:05:50.768426    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:06:04.871056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:04.871019    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:06:04.871488 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:04.871081    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:06:11.783475 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:11.783441    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:06:11.783938 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:11.783497    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:06:14.869937 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:14.869902    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:06:14.870300 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:14.869954    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:06:21.787052 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:21.784643    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:06:21.787052 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:21.784731    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:06:24.869337 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:24.869298    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:06:24.869734 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:24.869358    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:06:24.869734 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:24.869397    2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:06:24.869832 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:24.869815    2565 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"80ca4be251b3dab821b5db148af83b104c6aa0d7beaa58fb74e222b5fde38a68"} pod="openshift-image-registry/image-registry-787c4f4479-255zf" containerMessage="Container registry failed liveness probe, will be restarted"
Jul 09 12:06:24.873321 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:24.873296    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:06:24.873450 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:24.873336    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:06:34.874492 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:34.874455    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:06:34.874987 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:34.874527    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:06:44.874139 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:44.874103    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:06:44.874521 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:44.874198    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:06:49.889720 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:49.889658    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" containerID="cri-o://80ca4be251b3dab821b5db148af83b104c6aa0d7beaa58fb74e222b5fde38a68" gracePeriod=30
Jul 09 12:06:50.973436 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:50.973404    2565 generic.go:356] "Generic (PLEG): container finished" podID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerID="80ca4be251b3dab821b5db148af83b104c6aa0d7beaa58fb74e222b5fde38a68" exitCode=0
Jul 09 12:06:50.973841 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:50.973480    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerDied","Data":"80ca4be251b3dab821b5db148af83b104c6aa0d7beaa58fb74e222b5fde38a68"}
Jul 09 12:06:50.973841 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:50.973530    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerStarted","Data":"a686cc0f55af53da650d84d2c7c8fcf7d1200c56412caa11a5a6f7b7893f91b5"}
Jul 09 12:06:50.973841 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:50.973551    2565 scope.go:117] "RemoveContainer" containerID="6095e4afcab07de559504de336c73ea18399e7aba08d45808fa9058e3f71a7f5"
Jul 09 12:06:50.973841 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:06:50.973680    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:07:04.870990 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:04.870952    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:07:04.871429 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:04.871020    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:07:07.783513 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:07.783483    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:07:07.788595 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:07.788572    2565 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 12:07:07.788954 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:07.788940    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:07:11.983023 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:11.982987    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:07:11.985169 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:11.983037    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:07:14.869838 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:14.869794    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:07:14.870279 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:14.869868    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:07:21.983207 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:21.983176    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:07:21.983577 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:21.983247    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:07:24.869832 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:24.869798    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:07:24.870200 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:24.869848    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:07:24.870200 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:24.869886    2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:07:24.870319 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:24.870300    2565 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"a686cc0f55af53da650d84d2c7c8fcf7d1200c56412caa11a5a6f7b7893f91b5"} pod="openshift-image-registry/image-registry-787c4f4479-255zf" containerMessage="Container registry failed liveness probe, will be restarted"
Jul 09 12:07:24.873923 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:24.873899    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:07:24.874057 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:24.873935    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:07:34.874071 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:34.874038    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:07:34.874445 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:34.874090    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:07:44.874076 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:44.874044    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:07:44.874520 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:44.874100    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:07:49.889756 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:49.889718    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" containerID="cri-o://a686cc0f55af53da650d84d2c7c8fcf7d1200c56412caa11a5a6f7b7893f91b5" gracePeriod=30
Jul 09 12:07:50.175363 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:50.175290    2565 generic.go:356] "Generic (PLEG): container finished" podID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerID="a686cc0f55af53da650d84d2c7c8fcf7d1200c56412caa11a5a6f7b7893f91b5" exitCode=0
Jul 09 12:07:50.175363 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:50.175354    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerDied","Data":"a686cc0f55af53da650d84d2c7c8fcf7d1200c56412caa11a5a6f7b7893f91b5"}
Jul 09 12:07:50.175642 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:50.175396    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerStarted","Data":"be805d962a41ba7e241afd3d619ce5a5224aa0b39247844149f0d59d8d77e0d0"}
Jul 09 12:07:50.175642 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:50.175415    2565 scope.go:117] "RemoveContainer" containerID="80ca4be251b3dab821b5db148af83b104c6aa0d7beaa58fb74e222b5fde38a68"
Jul 09 12:07:50.175642 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:07:50.175586    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:08:04.871049 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:04.870967    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:08:04.871049 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:04.871020    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:08:11.185350 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:11.185312    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:08:11.185739 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:11.185372    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:08:14.869604 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:14.869572    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:08:14.869984 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:14.869622    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:08:21.184948 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:21.184916    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:08:21.185326 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:21.184964    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:08:24.869473 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:24.869435    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:08:24.869974 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:24.869506    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:08:24.869974 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:24.869552    2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:08:24.870133 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:24.870112    2565 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"be805d962a41ba7e241afd3d619ce5a5224aa0b39247844149f0d59d8d77e0d0"} pod="openshift-image-registry/image-registry-787c4f4479-255zf" containerMessage="Container registry failed liveness probe, will be restarted"
Jul 09 12:08:24.873779 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:24.873754    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:08:24.873890 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:24.873799    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:08:34.874217 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:34.874185    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:08:34.874584 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:34.874243    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:08:44.874350 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:44.874316    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:08:44.874808 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:44.874373    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:08:49.889370 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:49.889330    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" containerID="cri-o://be805d962a41ba7e241afd3d619ce5a5224aa0b39247844149f0d59d8d77e0d0" gracePeriod=30
Jul 09 12:08:50.372579 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:50.372550    2565 generic.go:356] "Generic (PLEG): container finished" podID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerID="be805d962a41ba7e241afd3d619ce5a5224aa0b39247844149f0d59d8d77e0d0" exitCode=0
Jul 09 12:08:50.372779 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:50.372622    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerDied","Data":"be805d962a41ba7e241afd3d619ce5a5224aa0b39247844149f0d59d8d77e0d0"}
Jul 09 12:08:50.372779 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:50.372675    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerStarted","Data":"78f975e6f3b137ea0cb768c15c8f5d30112be4f92770764fea61b970f50be434"}
Jul 09 12:08:50.372779 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:50.372695    2565 scope.go:117] "RemoveContainer" containerID="a686cc0f55af53da650d84d2c7c8fcf7d1200c56412caa11a5a6f7b7893f91b5"
Jul 09 12:08:50.372909 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:08:50.372816    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:09:04.871139 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:04.871096    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:09:04.871544 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:04.871156    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:09:11.383193 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:11.383158    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:09:11.383543 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:11.383209    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:09:14.869789 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:14.869737    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:09:14.870222 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:14.869814    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:09:21.383996 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:21.383954    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:09:21.384443 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:21.384015    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:09:24.869317 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:24.869284    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:09:24.869802 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:24.869347    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:09:24.869802 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:24.869391    2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:09:24.869989 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:24.869955    2565 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"78f975e6f3b137ea0cb768c15c8f5d30112be4f92770764fea61b970f50be434"} pod="openshift-image-registry/image-registry-787c4f4479-255zf" containerMessage="Container registry failed liveness probe, will be restarted"
Jul 09 12:09:24.873416 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:24.873394    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:09:24.873504 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:24.873442    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:09:34.873734 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:34.873687    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:09:34.874196 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:34.873759    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:09:44.873850 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:44.873815    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:09:44.874348 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:44.873871    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:09:49.889476 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:49.889423    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" containerID="cri-o://78f975e6f3b137ea0cb768c15c8f5d30112be4f92770764fea61b970f50be434" gracePeriod=30
Jul 09 12:09:50.005985 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:09:50.005959    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:09:50.571794 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:50.571766    2565 generic.go:356] "Generic (PLEG): container finished" podID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerID="78f975e6f3b137ea0cb768c15c8f5d30112be4f92770764fea61b970f50be434" exitCode=0
Jul 09 12:09:50.572046 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:50.571817    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerDied","Data":"78f975e6f3b137ea0cb768c15c8f5d30112be4f92770764fea61b970f50be434"}
Jul 09 12:09:50.572046 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:50.571859    2565 scope.go:117] "RemoveContainer" containerID="be805d962a41ba7e241afd3d619ce5a5224aa0b39247844149f0d59d8d77e0d0"
Jul 09 12:09:50.572246 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:09:50.572232    2565 scope.go:117] "RemoveContainer" containerID="78f975e6f3b137ea0cb768c15c8f5d30112be4f92770764fea61b970f50be434"
Jul 09 12:09:50.572419 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:09:50.572399    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:10:04.888613 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:10:04.888582    2565 scope.go:117] "RemoveContainer" containerID="78f975e6f3b137ea0cb768c15c8f5d30112be4f92770764fea61b970f50be434"
Jul 09 12:10:04.889102 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:04.888754    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:10:18.012982 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:18.012949    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Jul 09 12:10:18.012982 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:18.012986    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found
Jul 09 12:10:18.013456 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:18.013002    2565 projected.go:196] Error preparing data for projected volume kube-api-access-ltx9j for pod openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:10:18.013456 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:18.013072    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:10:18.513050412 +0000 UTC m=+491.190579458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ltx9j" (UniqueName: "kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:10:18.516755 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:18.516726    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Jul 09 12:10:18.516755 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:18.516755    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found
Jul 09 12:10:18.516755 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:18.516765    2565 projected.go:196] Error preparing data for projected volume kube-api-access-ltx9j for pod openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:10:18.516976 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:18.516813    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:10:19.516798687 +0000 UTC m=+492.194327723 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ltx9j" (UniqueName: "kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:10:19.524052 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:19.524018    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Jul 09 12:10:19.524052 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:19.524050    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found
Jul 09 12:10:19.524052 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:19.524060    2565 projected.go:196] Error preparing data for projected volume kube-api-access-ltx9j for pod openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:10:19.524478 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:19.524128    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:10:21.524113711 +0000 UTC m=+494.201642746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ltx9j" (UniqueName: "kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:10:19.888799 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:10:19.888769    2565 scope.go:117] "RemoveContainer" containerID="78f975e6f3b137ea0cb768c15c8f5d30112be4f92770764fea61b970f50be434"
Jul 09 12:10:19.888987 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:19.888955    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:10:21.536925 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:21.536892    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Jul 09 12:10:21.536925 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:21.536922    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found
Jul 09 12:10:21.536925 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:21.536931    2565 projected.go:196] Error preparing data for projected volume kube-api-access-ltx9j for pod openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:10:21.537348 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:21.536994    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:10:25.536975405 +0000 UTC m=+498.214504455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ltx9j" (UniqueName: "kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:10:25.569794 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:25.569757    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Jul 09 12:10:25.569794 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:25.569786    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found
Jul 09 12:10:25.569794 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:25.569795    2565 projected.go:196] Error preparing data for projected volume kube-api-access-ltx9j for pod openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:10:25.570322 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:25.569845    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:10:33.569830802 +0000 UTC m=+506.247359837 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ltx9j" (UniqueName: "kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:10:30.888208 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:10:30.888182    2565 scope.go:117] "RemoveContainer" containerID="78f975e6f3b137ea0cb768c15c8f5d30112be4f92770764fea61b970f50be434"
Jul 09 12:10:30.888664 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:30.888320    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:10:33.629553 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:33.629523    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Jul 09 12:10:33.629553 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:33.629554    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found
Jul 09 12:10:33.629553 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:33.629564    2565 projected.go:196] Error preparing data for projected volume kube-api-access-ltx9j for pod openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:10:33.629988 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:33.629614    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:10:49.629598231 +0000 UTC m=+522.307127287 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ltx9j" (UniqueName: "kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:10:45.888482 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:10:45.888451    2565 scope.go:117] "RemoveContainer" containerID="78f975e6f3b137ea0cb768c15c8f5d30112be4f92770764fea61b970f50be434"
Jul 09 12:10:45.888916 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:45.888625    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:10:49.645341 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:49.645306    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Jul 09 12:10:49.645341 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:49.645340    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found
Jul 09 12:10:49.645341 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:49.645350    2565 projected.go:196] Error preparing data for projected volume kube-api-access-ltx9j for pod openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:10:49.645846 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:49.645409    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:11:21.645395471 +0000 UTC m=+554.322924509 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ltx9j" (UniqueName: "kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:10:59.888452 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:10:59.888358    2565 scope.go:117] "RemoveContainer" containerID="78f975e6f3b137ea0cb768c15c8f5d30112be4f92770764fea61b970f50be434"
Jul 09 12:10:59.888928 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:10:59.888578    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:11:10.887849 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:10.887816    2565 scope.go:117] "RemoveContainer" containerID="78f975e6f3b137ea0cb768c15c8f5d30112be4f92770764fea61b970f50be434"
Jul 09 12:11:11.838138 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:11.838103    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerStarted","Data":"ac6b227c68497c0f7a3faf0f8d45b11b696d26015790bef5426b7a3b15ec5176"}
Jul 09 12:11:11.838330 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:11.838311    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:11:21.688418 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:11:21.688383    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Jul 09 12:11:21.688418 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:11:21.688412    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found
Jul 09 12:11:21.688418 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:11:21.688423    2565 projected.go:196] Error preparing data for projected volume kube-api-access-ltx9j for pod openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:11:21.688912 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:11:21.688473    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:12:25.688459051 +0000 UTC m=+618.365988087 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ltx9j" (UniqueName: "kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:11:24.871000 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:24.870958    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:11:24.871365 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:24.871017    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:11:32.845986 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:32.845948    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:11:32.846349 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:32.846006    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:11:34.869806 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:34.869770    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:11:34.870173 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:34.869826    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:11:42.846285 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:42.846240    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:11:42.846764 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:42.846300    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:11:44.869632 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:44.869597    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:11:44.870038 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:44.869655    2565 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:11:44.870038 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:44.869692    2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:11:44.870235 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:44.870213    2565 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"ac6b227c68497c0f7a3faf0f8d45b11b696d26015790bef5426b7a3b15ec5176"} pod="openshift-image-registry/image-registry-787c4f4479-255zf" containerMessage="Container registry failed liveness probe, will be restarted"
Jul 09 12:11:44.873931 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:44.873905    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:11:44.874051 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:44.873948    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:11:54.874100 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:54.874067    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:11:54.874480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:11:54.874140    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:12:04.873944 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:12:04.873901    2565 patch_prober.go:28] interesting pod/image-registry-787c4f4479-255zf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 12:12:04.874435 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:12:04.873970    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 12:12:07.804314 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:12:07.804287    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:12:07.807364 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:12:07.807340    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:12:09.889902 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:12:09.889859    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry" containerID="cri-o://ac6b227c68497c0f7a3faf0f8d45b11b696d26015790bef5426b7a3b15ec5176" gracePeriod=30
Jul 09 12:12:10.007647 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:12:10.007617    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:12:10.026535 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:12:10.026507    2565 generic.go:356] "Generic (PLEG): container finished" podID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerID="ac6b227c68497c0f7a3faf0f8d45b11b696d26015790bef5426b7a3b15ec5176" exitCode=0
Jul 09 12:12:10.026672 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:12:10.026578    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerDied","Data":"ac6b227c68497c0f7a3faf0f8d45b11b696d26015790bef5426b7a3b15ec5176"}
Jul 09 12:12:10.026672 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:12:10.026634    2565 scope.go:117] "RemoveContainer" containerID="78f975e6f3b137ea0cb768c15c8f5d30112be4f92770764fea61b970f50be434"
Jul 09 12:12:10.027017 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:12:10.026984    2565 scope.go:117] "RemoveContainer" containerID="ac6b227c68497c0f7a3faf0f8d45b11b696d26015790bef5426b7a3b15ec5176"
Jul 09 12:12:10.027188 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:12:10.027162    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:12:20.888975 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:12:20.888941    2565 scope.go:117] "RemoveContainer" containerID="ac6b227c68497c0f7a3faf0f8d45b11b696d26015790bef5426b7a3b15ec5176"
Jul 09 12:12:20.891257 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:12:20.889082    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:12:25.762131 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:12:25.762090    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found
Jul 09 12:12:25.762131 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:12:25.762125    2565 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found
Jul 09 12:12:25.762131 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:12:25.762138    2565 projected.go:196] Error preparing data for projected volume kube-api-access-ltx9j for pod openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dmclb: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:12:25.762767 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:12:25.762209    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j podName:165ea525-4d56-41f2-85b8-772b6abe82ff nodeName:}" failed. No retries permitted until 2026-07-09 12:14:27.762189556 +0000 UTC m=+740.439718611 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ltx9j" (UniqueName: "kubernetes.io/projected/165ea525-4d56-41f2-85b8-772b6abe82ff-kube-api-access-ltx9j") pod "cluster-samples-operator-56fcf86bf-dmclb" (UID: "165ea525-4d56-41f2-85b8-772b6abe82ff") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found]
Jul 09 12:12:35.888326 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:12:35.888297    2565 scope.go:117] "RemoveContainer" containerID="ac6b227c68497c0f7a3faf0f8d45b11b696d26015790bef5426b7a3b15ec5176"
Jul 09 12:12:35.888748 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:12:35.888434    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:12:50.888006 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:12:50.887975    2565 scope.go:117] "RemoveContainer" containerID="ac6b227c68497c0f7a3faf0f8d45b11b696d26015790bef5426b7a3b15ec5176"
Jul 09 12:12:50.888397 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:12:50.888137    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:13:02.888558 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:02.888526    2565 scope.go:117] "RemoveContainer" containerID="ac6b227c68497c0f7a3faf0f8d45b11b696d26015790bef5426b7a3b15ec5176"
Jul 09 12:13:02.889219 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:13:02.888675    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:13:16.887917 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:16.887888    2565 scope.go:117] "RemoveContainer" containerID="ac6b227c68497c0f7a3faf0f8d45b11b696d26015790bef5426b7a3b15ec5176"
Jul 09 12:13:16.888272 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:13:16.888047    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:13:28.888091 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:28.888057    2565 scope.go:117] "RemoveContainer" containerID="ac6b227c68497c0f7a3faf0f8d45b11b696d26015790bef5426b7a3b15ec5176"
Jul 09 12:13:28.888493 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:13:28.888209    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=registry pod=image-registry-787c4f4479-255zf_openshift-image-registry(87bb046d-1a4c-4eec-aa6b-812bdc1e5b21)\"" pod="openshift-image-registry/image-registry-787c4f4479-255zf" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"
Jul 09 12:13:29.366529 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.366498    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-787c4f4479-255zf"]
Jul 09 12:13:29.491819 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.491794    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:13:29.513241 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.513212    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls\") pod \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") "
Jul 09 12:13:29.513241 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.513246    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-bound-sa-token\") pod \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") "
Jul 09 12:13:29.513440 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.513268    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-installation-pull-secrets\") pod \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") "
Jul 09 12:13:29.513440 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.513302    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-certificates\") pod \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") "
Jul 09 12:13:29.513440 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.513317    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46x6k\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-kube-api-access-46x6k\") pod \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") "
Jul 09 12:13:29.513440 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.513341    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-image-registry-private-configuration\") pod \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") "
Jul 09 12:13:29.513440 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.513376    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-ca-trust-extracted\") pod \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") "
Jul 09 12:13:29.513440 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.513411    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-trusted-ca\") pod \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\" (UID: \"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21\") "
Jul 09 12:13:29.514296 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.513742    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" (UID: "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:13:29.514296 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.514160    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" (UID: "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:13:29.515891 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.515856    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" (UID: "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:13:29.516010 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.515956    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" (UID: "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:13:29.516158 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.516122    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-kube-api-access-46x6k" (OuterVolumeSpecName: "kube-api-access-46x6k") pod "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" (UID: "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"). InnerVolumeSpecName "kube-api-access-46x6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:13:29.516230 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.516192    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" (UID: "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:13:29.516306 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.516281    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" (UID: "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:13:29.524494 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.524465    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" (UID: "87bb046d-1a4c-4eec-aa6b-812bdc1e5b21"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:13:29.614549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.614513    2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-ca-trust-extracted\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:13:29.614549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.614540    2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-trusted-ca\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:13:29.614549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.614549    2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-tls\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:13:29.614549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.614557    2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-bound-sa-token\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:13:29.614831 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.614565    2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-installation-pull-secrets\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:13:29.614831 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.614573    2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-registry-certificates\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:13:29.614831 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.614580    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46x6k\" (UniqueName: \"kubernetes.io/projected/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-kube-api-access-46x6k\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:13:29.614831 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:29.614589    2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21-image-registry-private-configuration\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:13:30.292830 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:30.292750    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-787c4f4479-255zf"
Jul 09 12:13:30.292830 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:30.292754    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-787c4f4479-255zf" event={"ID":"87bb046d-1a4c-4eec-aa6b-812bdc1e5b21","Type":"ContainerDied","Data":"5483459b8ad94843a35b77217f237ed1b5242c964b1d55c002681b5d7d011620"}
Jul 09 12:13:30.292830 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:30.292825    2565 scope.go:117] "RemoveContainer" containerID="ac6b227c68497c0f7a3faf0f8d45b11b696d26015790bef5426b7a3b15ec5176"
Jul 09 12:13:30.311094 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:30.311069    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-787c4f4479-255zf"]
Jul 09 12:13:30.318295 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:30.318276    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-787c4f4479-255zf"]
Jul 09 12:13:31.891897 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:31.891868    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" path="/var/lib/kubelet/pods/87bb046d-1a4c-4eec-aa6b-812bdc1e5b21/volumes"
Jul 09 12:13:32.830100 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830069    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"]
Jul 09 12:13:32.830335 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830323    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830386 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830336    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830386 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830343    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830386 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830349    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830386 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830354    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830386 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830359    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830386 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830373    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830386 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830378    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830414    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830421    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830426    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830431    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830437    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830442    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830482    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830487    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830493    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830497    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830502    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830507    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.830609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.830536    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="87bb046d-1a4c-4eec-aa6b-812bdc1e5b21" containerName="registry"
Jul 09 12:13:32.833347 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.833332    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:32.836407 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.836384    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 09 12:13:32.837555 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.837541    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 09 12:13:32.837603 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.837561    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 12:13:32.837603 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.837572    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-r7s2d\""
Jul 09 12:13:32.844198 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.844176    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"]
Jul 09 12:13:32.940104 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.940073    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-vjp6w\" (UID: \"a88fb4f2-69ae-4746-ba6a-d38ca7e46986\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:32.940448 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.940140    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-vjp6w\" (UID: \"a88fb4f2-69ae-4746-ba6a-d38ca7e46986\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:32.940448 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.940162    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-vjp6w\" (UID: \"a88fb4f2-69ae-4746-ba6a-d38ca7e46986\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:32.940448 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:32.940189    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8s5k\" (UniqueName: \"kubernetes.io/projected/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-kube-api-access-g8s5k\") pod \"prometheus-operator-567f7b8b7c-vjp6w\" (UID: \"a88fb4f2-69ae-4746-ba6a-d38ca7e46986\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:33.041146 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:33.041116    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-vjp6w\" (UID: \"a88fb4f2-69ae-4746-ba6a-d38ca7e46986\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:33.041297 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:33.041162    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-vjp6w\" (UID: \"a88fb4f2-69ae-4746-ba6a-d38ca7e46986\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:33.041297 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:33.041187    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-vjp6w\" (UID: \"a88fb4f2-69ae-4746-ba6a-d38ca7e46986\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:33.041297 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:13:33.041280    2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 09 12:13:33.041400 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:33.041309    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8s5k\" (UniqueName: \"kubernetes.io/projected/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-kube-api-access-g8s5k\") pod \"prometheus-operator-567f7b8b7c-vjp6w\" (UID: \"a88fb4f2-69ae-4746-ba6a-d38ca7e46986\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:33.041400 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:13:33.041343    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-prometheus-operator-tls podName:a88fb4f2-69ae-4746-ba6a-d38ca7e46986 nodeName:}" failed. No retries permitted until 2026-07-09 12:13:33.541327183 +0000 UTC m=+686.218856219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-vjp6w" (UID: "a88fb4f2-69ae-4746-ba6a-d38ca7e46986") : secret "prometheus-operator-tls" not found
Jul 09 12:13:33.041884 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:33.041866    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-vjp6w\" (UID: \"a88fb4f2-69ae-4746-ba6a-d38ca7e46986\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:33.043620 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:33.043602    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-vjp6w\" (UID: \"a88fb4f2-69ae-4746-ba6a-d38ca7e46986\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:33.049923 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:33.049902    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8s5k\" (UniqueName: \"kubernetes.io/projected/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-kube-api-access-g8s5k\") pod \"prometheus-operator-567f7b8b7c-vjp6w\" (UID: \"a88fb4f2-69ae-4746-ba6a-d38ca7e46986\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:33.544024 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:33.543988    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-vjp6w\" (UID: \"a88fb4f2-69ae-4746-ba6a-d38ca7e46986\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:33.546451 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:33.546427    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a88fb4f2-69ae-4746-ba6a-d38ca7e46986-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-vjp6w\" (UID: \"a88fb4f2-69ae-4746-ba6a-d38ca7e46986\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:33.742141 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:33.742086    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"
Jul 09 12:13:33.870473 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:33.870442    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w"]
Jul 09 12:13:33.873160 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:13:33.873138    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda88fb4f2_69ae_4746_ba6a_d38ca7e46986.slice/crio-58e9dd84c18080b06d871d539d124263815ee0c833c74a90a82a7f7bb5c7f8c1 WatchSource:0}: Error finding container 58e9dd84c18080b06d871d539d124263815ee0c833c74a90a82a7f7bb5c7f8c1: Status 404 returned error can't find the container with id 58e9dd84c18080b06d871d539d124263815ee0c833c74a90a82a7f7bb5c7f8c1
Jul 09 12:13:33.875139 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:33.875123    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 12:13:34.307967 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:34.307882    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w" event={"ID":"a88fb4f2-69ae-4746-ba6a-d38ca7e46986","Type":"ContainerStarted","Data":"58e9dd84c18080b06d871d539d124263815ee0c833c74a90a82a7f7bb5c7f8c1"}
Jul 09 12:13:36.316112 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:36.316077    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w" event={"ID":"a88fb4f2-69ae-4746-ba6a-d38ca7e46986","Type":"ContainerStarted","Data":"66ffb17f64922596c275f78fb71aab59af3c5b16462c2b5db0404f16c66fe140"}
Jul 09 12:13:36.316469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:36.316132    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w" event={"ID":"a88fb4f2-69ae-4746-ba6a-d38ca7e46986","Type":"ContainerStarted","Data":"baa5c575fe41a0d668b9657fe43ddd5254ad0ea099ca925fb73f8e882aca4e4f"}
Jul 09 12:13:36.334120 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:36.334072    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-vjp6w" podStartSLOduration=2.800531456 podStartE2EDuration="4.334057264s" totalImagesPullingTime="1.533525808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:13:36.332959914 +0000 UTC m=+689.010488971" watchObservedRunningTime="2026-07-09 12:13:36.334057264 +0000 UTC m=+689.011586322"
Jul 09 12:13:38.175675 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.175645    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"]
Jul 09 12:13:38.178723 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.178688    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:38.183429 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.183404    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 09 12:13:38.183555 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.183452    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-2kvhc\""
Jul 09 12:13:38.183555 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.183506    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 09 12:13:38.202848 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.202828    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"]
Jul 09 12:13:38.220664 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.220636    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"]
Jul 09 12:13:38.223971 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.223952    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.229219 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.226582    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 09 12:13:38.229219 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.227846    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 09 12:13:38.229219 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.228392    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 09 12:13:38.239777 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.239760    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mk4w6\""
Jul 09 12:13:38.246969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.246948    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"]
Jul 09 12:13:38.254165 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.254143    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-slqgq"]
Jul 09 12:13:38.257091 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.257076    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.259631 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.259616    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5nslp\""
Jul 09 12:13:38.260091 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.260078    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 12:13:38.260265 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.260249    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 12:13:38.260545 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.260530    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 12:13:38.281134 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.281115    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e7f8647-20da-49b4-94b8-d55124ff3b94-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-s75s6\" (UID: \"7e7f8647-20da-49b4-94b8-d55124ff3b94\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:38.281215 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.281141    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b722a44f-8b60-4d9e-8a85-c699cb67f012-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.281215 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.281161    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r7zv\" (UniqueName: \"kubernetes.io/projected/b722a44f-8b60-4d9e-8a85-c699cb67f012-kube-api-access-7r7zv\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.281215 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.281180    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b722a44f-8b60-4d9e-8a85-c699cb67f012-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.281331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.281255    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e7f8647-20da-49b4-94b8-d55124ff3b94-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-s75s6\" (UID: \"7e7f8647-20da-49b4-94b8-d55124ff3b94\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:38.281331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.281307    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e7f8647-20da-49b4-94b8-d55124ff3b94-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-s75s6\" (UID: \"7e7f8647-20da-49b4-94b8-d55124ff3b94\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:38.281396 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.281327    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b722a44f-8b60-4d9e-8a85-c699cb67f012-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.281396 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.281352    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b722a44f-8b60-4d9e-8a85-c699cb67f012-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.281396 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.281370    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq7v7\" (UniqueName: \"kubernetes.io/projected/7e7f8647-20da-49b4-94b8-d55124ff3b94-kube-api-access-wq7v7\") pod \"openshift-state-metrics-54cf96dfcf-s75s6\" (UID: \"7e7f8647-20da-49b4-94b8-d55124ff3b94\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:38.281509 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.281407    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b722a44f-8b60-4d9e-8a85-c699cb67f012-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.382511 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.382479    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b722a44f-8b60-4d9e-8a85-c699cb67f012-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.382511 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.382512    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/85aaac54-cd86-4b4c-a917-25ab0245c451-root\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.382752 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.382527    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-tls\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.382752 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.382545    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85aaac54-cd86-4b4c-a917-25ab0245c451-metrics-client-ca\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.382752 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.382567    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e7f8647-20da-49b4-94b8-d55124ff3b94-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-s75s6\" (UID: \"7e7f8647-20da-49b4-94b8-d55124ff3b94\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:38.382752 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.382583    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b722a44f-8b60-4d9e-8a85-c699cb67f012-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.382752 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.382599    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-wtmp\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.382752 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:13:38.382669    2565 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 09 12:13:38.382752 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.382739    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r7zv\" (UniqueName: \"kubernetes.io/projected/b722a44f-8b60-4d9e-8a85-c699cb67f012-kube-api-access-7r7zv\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.383106 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:13:38.382762    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e7f8647-20da-49b4-94b8-d55124ff3b94-openshift-state-metrics-tls podName:7e7f8647-20da-49b4-94b8-d55124ff3b94 nodeName:}" failed. No retries permitted until 2026-07-09 12:13:38.882741476 +0000 UTC m=+691.560270518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7e7f8647-20da-49b4-94b8-d55124ff3b94-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-s75s6" (UID: "7e7f8647-20da-49b4-94b8-d55124ff3b94") : secret "openshift-state-metrics-tls" not found
Jul 09 12:13:38.383106 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.382801    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b722a44f-8b60-4d9e-8a85-c699cb67f012-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.383106 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.382838    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85aaac54-cd86-4b4c-a917-25ab0245c451-sys\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.383106 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.382862    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-textfile\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.383106 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.382890    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e7f8647-20da-49b4-94b8-d55124ff3b94-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-s75s6\" (UID: \"7e7f8647-20da-49b4-94b8-d55124ff3b94\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:38.383106 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.382977    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e7f8647-20da-49b4-94b8-d55124ff3b94-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-s75s6\" (UID: \"7e7f8647-20da-49b4-94b8-d55124ff3b94\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:38.383106 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.383014    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.383106 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.383046    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b722a44f-8b60-4d9e-8a85-c699cb67f012-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.383106 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.383079    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b722a44f-8b60-4d9e-8a85-c699cb67f012-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.383746 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.383114    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-accelerators-collector-config\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.383746 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.383147    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq7v7\" (UniqueName: \"kubernetes.io/projected/7e7f8647-20da-49b4-94b8-d55124ff3b94-kube-api-access-wq7v7\") pod \"openshift-state-metrics-54cf96dfcf-s75s6\" (UID: \"7e7f8647-20da-49b4-94b8-d55124ff3b94\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:38.383746 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.383156    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b722a44f-8b60-4d9e-8a85-c699cb67f012-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.383746 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.383175    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdfsw\" (UniqueName: \"kubernetes.io/projected/85aaac54-cd86-4b4c-a917-25ab0245c451-kube-api-access-vdfsw\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.383958 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.383753    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b722a44f-8b60-4d9e-8a85-c699cb67f012-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.383958 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.383897    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b722a44f-8b60-4d9e-8a85-c699cb67f012-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.383958 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.383910    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e7f8647-20da-49b4-94b8-d55124ff3b94-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-s75s6\" (UID: \"7e7f8647-20da-49b4-94b8-d55124ff3b94\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:38.385297 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.385271    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e7f8647-20da-49b4-94b8-d55124ff3b94-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-s75s6\" (UID: \"7e7f8647-20da-49b4-94b8-d55124ff3b94\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:38.385387 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.385323    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b722a44f-8b60-4d9e-8a85-c699cb67f012-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.385387 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.385355    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b722a44f-8b60-4d9e-8a85-c699cb67f012-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.390885 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.390865    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq7v7\" (UniqueName: \"kubernetes.io/projected/7e7f8647-20da-49b4-94b8-d55124ff3b94-kube-api-access-wq7v7\") pod \"openshift-state-metrics-54cf96dfcf-s75s6\" (UID: \"7e7f8647-20da-49b4-94b8-d55124ff3b94\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:38.391270 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.391244    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r7zv\" (UniqueName: \"kubernetes.io/projected/b722a44f-8b60-4d9e-8a85-c699cb67f012-kube-api-access-7r7zv\") pod \"kube-state-metrics-7c9dbb8b7d-bbrvx\" (UID: \"b722a44f-8b60-4d9e-8a85-c699cb67f012\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.484353 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.484283    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-wtmp\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.484353 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.484322    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85aaac54-cd86-4b4c-a917-25ab0245c451-sys\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.484353 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.484346    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-textfile\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.484618 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.484389    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.484618 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.484397    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85aaac54-cd86-4b4c-a917-25ab0245c451-sys\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.484618 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.484425    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-accelerators-collector-config\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.484618 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.484454    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdfsw\" (UniqueName: \"kubernetes.io/projected/85aaac54-cd86-4b4c-a917-25ab0245c451-kube-api-access-vdfsw\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.484618 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.484477    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-wtmp\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.484618 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.484490    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/85aaac54-cd86-4b4c-a917-25ab0245c451-root\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.484618 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.484530    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/85aaac54-cd86-4b4c-a917-25ab0245c451-root\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.484618 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.484545    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-tls\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.484946 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.484677    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85aaac54-cd86-4b4c-a917-25ab0245c451-metrics-client-ca\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.484946 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.484742    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-textfile\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.485112 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.485091    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-accelerators-collector-config\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.485153 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.485135    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85aaac54-cd86-4b4c-a917-25ab0245c451-metrics-client-ca\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.486735 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.486692    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.486877 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.486860    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/85aaac54-cd86-4b4c-a917-25ab0245c451-node-exporter-tls\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.492724 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.492678    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdfsw\" (UniqueName: \"kubernetes.io/projected/85aaac54-cd86-4b4c-a917-25ab0245c451-kube-api-access-vdfsw\") pod \"node-exporter-slqgq\" (UID: \"85aaac54-cd86-4b4c-a917-25ab0245c451\") " pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.536607 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.536590    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"
Jul 09 12:13:38.565897 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.565875    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-slqgq"
Jul 09 12:13:38.576545 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:13:38.576518    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85aaac54_cd86_4b4c_a917_25ab0245c451.slice/crio-0fa925fe269cd27865f25973c8f613f4aae2642d8369d28510ee967c381b70da WatchSource:0}: Error finding container 0fa925fe269cd27865f25973c8f613f4aae2642d8369d28510ee967c381b70da: Status 404 returned error can't find the container with id 0fa925fe269cd27865f25973c8f613f4aae2642d8369d28510ee967c381b70da
Jul 09 12:13:38.666235 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.666171    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx"]
Jul 09 12:13:38.668586 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:13:38.668563    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb722a44f_8b60_4d9e_8a85_c699cb67f012.slice/crio-f668cad558427189b3a8e6779c645a94bee48f8dd7d202adcce63865cc22993a WatchSource:0}: Error finding container f668cad558427189b3a8e6779c645a94bee48f8dd7d202adcce63865cc22993a: Status 404 returned error can't find the container with id f668cad558427189b3a8e6779c645a94bee48f8dd7d202adcce63865cc22993a
Jul 09 12:13:38.888837 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.888806    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e7f8647-20da-49b4-94b8-d55124ff3b94-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-s75s6\" (UID: \"7e7f8647-20da-49b4-94b8-d55124ff3b94\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:38.891207 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:38.891189    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e7f8647-20da-49b4-94b8-d55124ff3b94-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-s75s6\" (UID: \"7e7f8647-20da-49b4-94b8-d55124ff3b94\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:39.086878 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:39.086846    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"
Jul 09 12:13:39.224402 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:39.224369    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6"]
Jul 09 12:13:39.227715 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:13:39.227676    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e7f8647_20da_49b4_94b8_d55124ff3b94.slice/crio-eda774ecb295274e35d52f9425ccbd63e2ce5ecbca1047fa98b41e832aa62e07 WatchSource:0}: Error finding container eda774ecb295274e35d52f9425ccbd63e2ce5ecbca1047fa98b41e832aa62e07: Status 404 returned error can't find the container with id eda774ecb295274e35d52f9425ccbd63e2ce5ecbca1047fa98b41e832aa62e07
Jul 09 12:13:39.334402 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:39.334328    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-slqgq" event={"ID":"85aaac54-cd86-4b4c-a917-25ab0245c451","Type":"ContainerStarted","Data":"0fa925fe269cd27865f25973c8f613f4aae2642d8369d28510ee967c381b70da"}
Jul 09 12:13:39.338958 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:39.338857    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6" event={"ID":"7e7f8647-20da-49b4-94b8-d55124ff3b94","Type":"ContainerStarted","Data":"4ad3e996706d51c6199ff2c852d6d0cd87cb6a3087a74aa457270e7878cc5fdd"}
Jul 09 12:13:39.338958 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:39.338920    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6" event={"ID":"7e7f8647-20da-49b4-94b8-d55124ff3b94","Type":"ContainerStarted","Data":"eda774ecb295274e35d52f9425ccbd63e2ce5ecbca1047fa98b41e832aa62e07"}
Jul 09 12:13:39.342677 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:39.342647    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx" event={"ID":"b722a44f-8b60-4d9e-8a85-c699cb67f012","Type":"ContainerStarted","Data":"f668cad558427189b3a8e6779c645a94bee48f8dd7d202adcce63865cc22993a"}
Jul 09 12:13:40.350808 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:40.350769    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-slqgq" event={"ID":"85aaac54-cd86-4b4c-a917-25ab0245c451","Type":"ContainerStarted","Data":"8866ceede6da095aac9ae5845b5ae17164968665995c9e6181fe64de240cb301"}
Jul 09 12:13:40.354441 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:40.354351    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6" event={"ID":"7e7f8647-20da-49b4-94b8-d55124ff3b94","Type":"ContainerStarted","Data":"78bde2cd9c1ba0f97bce0767d7852476821cba9aa33df8cd59dbdbc280a61669"}
Jul 09 12:13:40.358277 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:40.358222    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx" event={"ID":"b722a44f-8b60-4d9e-8a85-c699cb67f012","Type":"ContainerStarted","Data":"ea6bb3167fd1d701b762badfa2a85a3747d1e66704749ce6e60a05637017fda7"}
Jul 09 12:13:40.358394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:40.358282    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx" event={"ID":"b722a44f-8b60-4d9e-8a85-c699cb67f012","Type":"ContainerStarted","Data":"3eb1a0fb407c7586099eba6f8b2582ed1ad5b95ba5b9aa8604d1f08c69f843f1"}
Jul 09 12:13:41.364477 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:41.364434    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6" event={"ID":"7e7f8647-20da-49b4-94b8-d55124ff3b94","Type":"ContainerStarted","Data":"135449e54302cd3aad4ebbbaea65bfefbc40ef8ca1257eb54e577ce3fb4768ec"}
Jul 09 12:13:41.366493 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:41.366459    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx" event={"ID":"b722a44f-8b60-4d9e-8a85-c699cb67f012","Type":"ContainerStarted","Data":"cfa048c790bf7ff8e0a62ea1cf6afa11af6b95fb876edf6d1d8c61fc1961448d"}
Jul 09 12:13:41.367910 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:41.367859    2565 generic.go:356] "Generic (PLEG): container finished" podID="85aaac54-cd86-4b4c-a917-25ab0245c451" containerID="8866ceede6da095aac9ae5845b5ae17164968665995c9e6181fe64de240cb301" exitCode=0
Jul 09 12:13:41.367910 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:41.367896    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-slqgq" event={"ID":"85aaac54-cd86-4b4c-a917-25ab0245c451","Type":"ContainerDied","Data":"8866ceede6da095aac9ae5845b5ae17164968665995c9e6181fe64de240cb301"}
Jul 09 12:13:41.383001 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:41.382959    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-s75s6" podStartSLOduration=1.851649401 podStartE2EDuration="3.382944226s" totalImagesPullingTime="1.531294825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:13:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:13:41.381840199 +0000 UTC m=+694.059369258" watchObservedRunningTime="2026-07-09 12:13:41.382944226 +0000 UTC m=+694.060473284"
Jul 09 12:13:41.400132 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:41.399992    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-bbrvx" podStartSLOduration=1.885777845 podStartE2EDuration="3.399971143s" totalImagesPullingTime="1.514193298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:13:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:13:41.398438887 +0000 UTC m=+694.075967957" watchObservedRunningTime="2026-07-09 12:13:41.399971143 +0000 UTC m=+694.077500201"
Jul 09 12:13:42.374403 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.374366    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-slqgq" event={"ID":"85aaac54-cd86-4b4c-a917-25ab0245c451","Type":"ContainerStarted","Data":"0daca9c9fbc995287e2651a912a55436dd0920faf3ff1e99d9074a9adb4f2ba4"}
Jul 09 12:13:42.374808 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.374433    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-slqgq" event={"ID":"85aaac54-cd86-4b4c-a917-25ab0245c451","Type":"ContainerStarted","Data":"d0c81f72cf90b32b2e29b1c2b3149a8595bfe96f8f47faf02f367682b0af65ad"}
Jul 09 12:13:42.403870 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.403819    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-slqgq" podStartSLOduration=2.699306085 podStartE2EDuration="4.403801064s" totalImagesPullingTime="1.603997113s" totalInitContainerRuntime="100.497866ms" isStatefulPod=true podCreationTimestamp="2026-07-09 12:13:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:13:42.402029862 +0000 UTC m=+695.079558919" watchObservedRunningTime="2026-07-09 12:13:42.403801064 +0000 UTC m=+695.081330123"
Jul 09 12:13:42.558828 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.558795    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"]
Jul 09 12:13:42.562041 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.562019    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.565438 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.565416    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bhb11jhj5nok1\""
Jul 09 12:13:42.565438 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.565437    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 09 12:13:42.565602 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.565439    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 09 12:13:42.565662 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.565607    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-zx6z2\""
Jul 09 12:13:42.565899 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.565884    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 09 12:13:42.565955 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.565922    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 09 12:13:42.580865 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.580838    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"]
Jul 09 12:13:42.623882 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.623847    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c25ml\" (UniqueName: \"kubernetes.io/projected/81fab021-35cd-4aa1-abc6-be93fe07be1a-kube-api-access-c25ml\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.623882 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.623891    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/81fab021-35cd-4aa1-abc6-be93fe07be1a-secret-metrics-server-tls\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.624116 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.623964    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/81fab021-35cd-4aa1-abc6-be93fe07be1a-secret-metrics-server-client-certs\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.624116 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.624013    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/81fab021-35cd-4aa1-abc6-be93fe07be1a-audit-log\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.624116 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.624050    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/81fab021-35cd-4aa1-abc6-be93fe07be1a-client-ca-bundle\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.624116 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.624071    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/81fab021-35cd-4aa1-abc6-be93fe07be1a-metrics-server-audit-profiles\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.624116 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.624093    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81fab021-35cd-4aa1-abc6-be93fe07be1a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.725143 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.725039    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c25ml\" (UniqueName: \"kubernetes.io/projected/81fab021-35cd-4aa1-abc6-be93fe07be1a-kube-api-access-c25ml\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.725143 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.725095    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/81fab021-35cd-4aa1-abc6-be93fe07be1a-secret-metrics-server-tls\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.725143 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.725144    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/81fab021-35cd-4aa1-abc6-be93fe07be1a-secret-metrics-server-client-certs\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.725406 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.725176    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/81fab021-35cd-4aa1-abc6-be93fe07be1a-audit-log\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.725406 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.725369    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/81fab021-35cd-4aa1-abc6-be93fe07be1a-client-ca-bundle\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.725492 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.725410    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/81fab021-35cd-4aa1-abc6-be93fe07be1a-metrics-server-audit-profiles\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.725492 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.725451    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81fab021-35cd-4aa1-abc6-be93fe07be1a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.725672 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.725647    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/81fab021-35cd-4aa1-abc6-be93fe07be1a-audit-log\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.726188 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.726148    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81fab021-35cd-4aa1-abc6-be93fe07be1a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.726798 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.726774    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/81fab021-35cd-4aa1-abc6-be93fe07be1a-metrics-server-audit-profiles\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.727800 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.727779    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/81fab021-35cd-4aa1-abc6-be93fe07be1a-secret-metrics-server-tls\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.727878 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.727867    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/81fab021-35cd-4aa1-abc6-be93fe07be1a-client-ca-bundle\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.728064 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.728043    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/81fab021-35cd-4aa1-abc6-be93fe07be1a-secret-metrics-server-client-certs\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.733754 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.733732    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c25ml\" (UniqueName: \"kubernetes.io/projected/81fab021-35cd-4aa1-abc6-be93fe07be1a-kube-api-access-c25ml\") pod \"metrics-server-5f76b8b7c6-gmhqp\" (UID: \"81fab021-35cd-4aa1-abc6-be93fe07be1a\") " pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:42.871103 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:42.871071    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:13:43.024400 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:43.024362    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"]
Jul 09 12:13:43.026985 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:13:43.026955    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81fab021_35cd_4aa1_abc6_be93fe07be1a.slice/crio-9be261e1cba5414a8c8efb98c5b377e54de24a1f9ae6c646133aee0a1047a1d3 WatchSource:0}: Error finding container 9be261e1cba5414a8c8efb98c5b377e54de24a1f9ae6c646133aee0a1047a1d3: Status 404 returned error can't find the container with id 9be261e1cba5414a8c8efb98c5b377e54de24a1f9ae6c646133aee0a1047a1d3
Jul 09 12:13:43.380378 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:43.380333    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp" event={"ID":"81fab021-35cd-4aa1-abc6-be93fe07be1a","Type":"ContainerStarted","Data":"9be261e1cba5414a8c8efb98c5b377e54de24a1f9ae6c646133aee0a1047a1d3"}
Jul 09 12:13:45.389534 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:45.389495    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp" event={"ID":"81fab021-35cd-4aa1-abc6-be93fe07be1a","Type":"ContainerStarted","Data":"adc574a7c581e219f9c174f589368db2d58b89258ea8aaead3ffc6edb3029d48"}
Jul 09 12:13:45.409002 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:13:45.408954    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp" podStartSLOduration=1.446177345 podStartE2EDuration="3.408938713s" totalImagesPullingTime="1.962761368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:13:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:13:45.406908149 +0000 UTC m=+698.084437207" watchObservedRunningTime="2026-07-09 12:13:45.408938713 +0000 UTC m=+698.086467771"
Jul 09 12:14:02.871444 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:14:02.871345    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:14:02.871444 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:14:02.871426    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:14:22.876645 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:14:22.876613    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:14:22.880281 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:14:22.880260    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5f76b8b7c6-gmhqp"
Jul 09 12:16:23.697616 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.697581    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db8696df6-rgh5s"]
Jul 09 12:16:23.700571 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.700556    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db8696df6-rgh5s"
Jul 09 12:16:23.704373 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.704344    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 09 12:16:23.704515 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.704388    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 09 12:16:23.704515 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.704391    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-rqpcj\""
Jul 09 12:16:23.704515 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.704440    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 09 12:16:23.704515 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.704441    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 09 12:16:23.710882 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.710861    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db8696df6-rgh5s"]
Jul 09 12:16:23.767818 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.767783    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"]
Jul 09 12:16:23.770931 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.770914    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"
Jul 09 12:16:23.773564 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.773547    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 09 12:16:23.782189 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.782166    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"]
Jul 09 12:16:23.821899 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.821866    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d9a7e9df-0fa0-4923-a700-afa8e8d1339a-klusterlet-config\") pod \"klusterlet-addon-workmgr-778b48457-nxzg6\" (UID: \"d9a7e9df-0fa0-4923-a700-afa8e8d1339a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"
Jul 09 12:16:23.821899 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.821899    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9a7e9df-0fa0-4923-a700-afa8e8d1339a-tmp\") pod \"klusterlet-addon-workmgr-778b48457-nxzg6\" (UID: \"d9a7e9df-0fa0-4923-a700-afa8e8d1339a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"
Jul 09 12:16:23.822098 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.821920    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtsjd\" (UniqueName: \"kubernetes.io/projected/d9a7e9df-0fa0-4923-a700-afa8e8d1339a-kube-api-access-xtsjd\") pod \"klusterlet-addon-workmgr-778b48457-nxzg6\" (UID: \"d9a7e9df-0fa0-4923-a700-afa8e8d1339a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"
Jul 09 12:16:23.822098 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.821958    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e389969-bf18-436d-a91d-f9b2369ea952-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7db8696df6-rgh5s\" (UID: \"6e389969-bf18-436d-a91d-f9b2369ea952\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db8696df6-rgh5s"
Jul 09 12:16:23.822098 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.822009    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmtbj\" (UniqueName: \"kubernetes.io/projected/6e389969-bf18-436d-a91d-f9b2369ea952-kube-api-access-fmtbj\") pod \"managed-serviceaccount-addon-agent-7db8696df6-rgh5s\" (UID: \"6e389969-bf18-436d-a91d-f9b2369ea952\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db8696df6-rgh5s"
Jul 09 12:16:23.922982 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.922942    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d9a7e9df-0fa0-4923-a700-afa8e8d1339a-klusterlet-config\") pod \"klusterlet-addon-workmgr-778b48457-nxzg6\" (UID: \"d9a7e9df-0fa0-4923-a700-afa8e8d1339a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"
Jul 09 12:16:23.922982 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.922980    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9a7e9df-0fa0-4923-a700-afa8e8d1339a-tmp\") pod \"klusterlet-addon-workmgr-778b48457-nxzg6\" (UID: \"d9a7e9df-0fa0-4923-a700-afa8e8d1339a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"
Jul 09 12:16:23.923202 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.922999    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtsjd\" (UniqueName: \"kubernetes.io/projected/d9a7e9df-0fa0-4923-a700-afa8e8d1339a-kube-api-access-xtsjd\") pod \"klusterlet-addon-workmgr-778b48457-nxzg6\" (UID: \"d9a7e9df-0fa0-4923-a700-afa8e8d1339a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"
Jul 09 12:16:23.923202 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.923036    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e389969-bf18-436d-a91d-f9b2369ea952-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7db8696df6-rgh5s\" (UID: \"6e389969-bf18-436d-a91d-f9b2369ea952\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db8696df6-rgh5s"
Jul 09 12:16:23.923202 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.923080    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmtbj\" (UniqueName: \"kubernetes.io/projected/6e389969-bf18-436d-a91d-f9b2369ea952-kube-api-access-fmtbj\") pod \"managed-serviceaccount-addon-agent-7db8696df6-rgh5s\" (UID: \"6e389969-bf18-436d-a91d-f9b2369ea952\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db8696df6-rgh5s"
Jul 09 12:16:23.923366 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.923315    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d9a7e9df-0fa0-4923-a700-afa8e8d1339a-tmp\") pod \"klusterlet-addon-workmgr-778b48457-nxzg6\" (UID: \"d9a7e9df-0fa0-4923-a700-afa8e8d1339a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"
Jul 09 12:16:23.925632 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.925609    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e389969-bf18-436d-a91d-f9b2369ea952-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7db8696df6-rgh5s\" (UID: \"6e389969-bf18-436d-a91d-f9b2369ea952\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db8696df6-rgh5s"
Jul 09 12:16:23.925632 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.925624    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d9a7e9df-0fa0-4923-a700-afa8e8d1339a-klusterlet-config\") pod \"klusterlet-addon-workmgr-778b48457-nxzg6\" (UID: \"d9a7e9df-0fa0-4923-a700-afa8e8d1339a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"
Jul 09 12:16:23.931435 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.931412    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmtbj\" (UniqueName: \"kubernetes.io/projected/6e389969-bf18-436d-a91d-f9b2369ea952-kube-api-access-fmtbj\") pod \"managed-serviceaccount-addon-agent-7db8696df6-rgh5s\" (UID: \"6e389969-bf18-436d-a91d-f9b2369ea952\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db8696df6-rgh5s"
Jul 09 12:16:23.932137 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:23.932110    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtsjd\" (UniqueName: \"kubernetes.io/projected/d9a7e9df-0fa0-4923-a700-afa8e8d1339a-kube-api-access-xtsjd\") pod \"klusterlet-addon-workmgr-778b48457-nxzg6\" (UID: \"d9a7e9df-0fa0-4923-a700-afa8e8d1339a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"
Jul 09 12:16:24.018395 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:24.018329    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db8696df6-rgh5s"
Jul 09 12:16:24.080491 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:24.080463    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"
Jul 09 12:16:24.156970 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:24.156942    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db8696df6-rgh5s"]
Jul 09 12:16:24.160134 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:16:24.160109    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e389969_bf18_436d_a91d_f9b2369ea952.slice/crio-1b4c329d2492643e3f5ff7cef81996d05bda12d93e60dcf1eb5cb36dddc09558 WatchSource:0}: Error finding container 1b4c329d2492643e3f5ff7cef81996d05bda12d93e60dcf1eb5cb36dddc09558: Status 404 returned error can't find the container with id 1b4c329d2492643e3f5ff7cef81996d05bda12d93e60dcf1eb5cb36dddc09558
Jul 09 12:16:24.214514 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:24.214484    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"]
Jul 09 12:16:24.217716 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:16:24.217673    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a7e9df_0fa0_4923_a700_afa8e8d1339a.slice/crio-f6f5ad1775fc4ae4a58450663d09c318a3dfe6984185263d31467ec3c148c9e1 WatchSource:0}: Error finding container f6f5ad1775fc4ae4a58450663d09c318a3dfe6984185263d31467ec3c148c9e1: Status 404 returned error can't find the container with id f6f5ad1775fc4ae4a58450663d09c318a3dfe6984185263d31467ec3c148c9e1
Jul 09 12:16:24.968743 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:24.968662    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db8696df6-rgh5s" event={"ID":"6e389969-bf18-436d-a91d-f9b2369ea952","Type":"ContainerStarted","Data":"1b4c329d2492643e3f5ff7cef81996d05bda12d93e60dcf1eb5cb36dddc09558"}
Jul 09 12:16:24.970252 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:24.970221    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6" event={"ID":"d9a7e9df-0fa0-4923-a700-afa8e8d1339a","Type":"ContainerStarted","Data":"f6f5ad1775fc4ae4a58450663d09c318a3dfe6984185263d31467ec3c148c9e1"}
Jul 09 12:16:26.024164 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.024123    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"]
Jul 09 12:16:26.036970 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.036939    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"
Jul 09 12:16:26.037597 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.037534    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"]
Jul 09 12:16:26.039836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.039570    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v46hf\""
Jul 09 12:16:26.039836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.039570    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 12:16:26.040940 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.040883    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 12:16:26.141270 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.141227    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn\" (UID: \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"
Jul 09 12:16:26.141449 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.141315    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn\" (UID: \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"
Jul 09 12:16:26.141449 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.141358    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gfgf\" (UniqueName: \"kubernetes.io/projected/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-kube-api-access-7gfgf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn\" (UID: \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"
Jul 09 12:16:26.242189 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.242157    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn\" (UID: \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"
Jul 09 12:16:26.242381 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.242222    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn\" (UID: \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"
Jul 09 12:16:26.242381 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.242254    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gfgf\" (UniqueName: \"kubernetes.io/projected/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-kube-api-access-7gfgf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn\" (UID: \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"
Jul 09 12:16:26.242636 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.242604    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn\" (UID: \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"
Jul 09 12:16:26.242797 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.242654    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn\" (UID: \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"
Jul 09 12:16:26.251429 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.251397    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gfgf\" (UniqueName: \"kubernetes.io/projected/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-kube-api-access-7gfgf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn\" (UID: \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"
Jul 09 12:16:26.350955 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:26.350917    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"
Jul 09 12:16:27.555246 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:27.555202    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"]
Jul 09 12:16:27.557593 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:16:27.557565    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc5eef8_d251_408c_9a91_e2d0f9ee9799.slice/crio-d120e3ed0920bf444344a2223f9973fa8a17db8430660e2c5d93c019b0fed102 WatchSource:0}: Error finding container d120e3ed0920bf444344a2223f9973fa8a17db8430660e2c5d93c019b0fed102: Status 404 returned error can't find the container with id d120e3ed0920bf444344a2223f9973fa8a17db8430660e2c5d93c019b0fed102
Jul 09 12:16:27.983059 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:27.983026    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6" event={"ID":"d9a7e9df-0fa0-4923-a700-afa8e8d1339a","Type":"ContainerStarted","Data":"ae26020f3ca700e45e6c26ef9e3d1f0cd7a72629969b007c736da008a6f9c51d"}
Jul 09 12:16:27.983231 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:27.983219    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"
Jul 09 12:16:27.984409 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:27.984383    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn" event={"ID":"9bc5eef8-d251-408c-9a91-e2d0f9ee9799","Type":"ContainerStarted","Data":"d120e3ed0920bf444344a2223f9973fa8a17db8430660e2c5d93c019b0fed102"}
Jul 09 12:16:27.985165 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:27.985147    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6"
Jul 09 12:16:28.001098 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:28.001050    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-778b48457-nxzg6" podStartSLOduration=1.719877213 podStartE2EDuration="5.001035337s" totalImagesPullingTime="3.281158124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:16:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:16:28.000175674 +0000 UTC m=+860.677704754" watchObservedRunningTime="2026-07-09 12:16:28.001035337 +0000 UTC m=+860.678564392"
Jul 09 12:16:29.994622 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:29.994584    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db8696df6-rgh5s" event={"ID":"6e389969-bf18-436d-a91d-f9b2369ea952","Type":"ContainerStarted","Data":"2dc7470ed3bfb8edaca480aec0fed4e8db9bb63afcecc60d05a21805cd431ab8"}
Jul 09 12:16:30.019579 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:30.019531    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7db8696df6-rgh5s" podStartSLOduration=2.016663558 podStartE2EDuration="7.019515732s" totalImagesPullingTime="5.002852174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:16:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:16:30.018065851 +0000 UTC m=+862.695594905" watchObservedRunningTime="2026-07-09 12:16:30.019515732 +0000 UTC m=+862.697044790"
Jul 09 12:16:37.025280 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:37.025243    2565 generic.go:356] "Generic (PLEG): container finished" podID="9bc5eef8-d251-408c-9a91-e2d0f9ee9799" containerID="183c44701fc1f5ed9a2f9e06930f5df4c056c0a47c98372dd7168abf14055db8" exitCode=0
Jul 09 12:16:37.025739 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:37.025326    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn" event={"ID":"9bc5eef8-d251-408c-9a91-e2d0f9ee9799","Type":"ContainerDied","Data":"183c44701fc1f5ed9a2f9e06930f5df4c056c0a47c98372dd7168abf14055db8"}
Jul 09 12:16:40.038518 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:40.038482    2565 generic.go:356] "Generic (PLEG): container finished" podID="9bc5eef8-d251-408c-9a91-e2d0f9ee9799" containerID="ec6aec11d3b287f764721ecb031238590abc2425ec9a2d913126c919380f9db3" exitCode=0
Jul 09 12:16:40.038949 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:40.038568    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn" event={"ID":"9bc5eef8-d251-408c-9a91-e2d0f9ee9799","Type":"ContainerDied","Data":"ec6aec11d3b287f764721ecb031238590abc2425ec9a2d913126c919380f9db3"}
Jul 09 12:16:49.076612 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:49.076576    2565 generic.go:356] "Generic (PLEG): container finished" podID="9bc5eef8-d251-408c-9a91-e2d0f9ee9799" containerID="bfba65e482628a132082be58f623a00059e2bec1e1ccdf49c3a36fdfe8468be2" exitCode=0
Jul 09 12:16:49.077022 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:49.076651    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn" event={"ID":"9bc5eef8-d251-408c-9a91-e2d0f9ee9799","Type":"ContainerDied","Data":"bfba65e482628a132082be58f623a00059e2bec1e1ccdf49c3a36fdfe8468be2"}
Jul 09 12:16:50.208340 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:50.208311    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"
Jul 09 12:16:50.349104 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:50.349021    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-util\") pod \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\" (UID: \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\") "
Jul 09 12:16:50.349104 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:50.349072    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gfgf\" (UniqueName: \"kubernetes.io/projected/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-kube-api-access-7gfgf\") pod \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\" (UID: \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\") "
Jul 09 12:16:50.349315 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:50.349161    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-bundle\") pod \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\" (UID: \"9bc5eef8-d251-408c-9a91-e2d0f9ee9799\") "
Jul 09 12:16:50.349850 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:50.349826    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-bundle" (OuterVolumeSpecName: "bundle") pod "9bc5eef8-d251-408c-9a91-e2d0f9ee9799" (UID: "9bc5eef8-d251-408c-9a91-e2d0f9ee9799"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:16:50.351453 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:50.351429    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-kube-api-access-7gfgf" (OuterVolumeSpecName: "kube-api-access-7gfgf") pod "9bc5eef8-d251-408c-9a91-e2d0f9ee9799" (UID: "9bc5eef8-d251-408c-9a91-e2d0f9ee9799"). InnerVolumeSpecName "kube-api-access-7gfgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:16:50.353604 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:50.353581    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-util" (OuterVolumeSpecName: "util") pod "9bc5eef8-d251-408c-9a91-e2d0f9ee9799" (UID: "9bc5eef8-d251-408c-9a91-e2d0f9ee9799"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:16:50.450399 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:50.450369    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-bundle\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:16:50.450399 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:50.450395    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-util\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:16:50.450399 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:50.450405    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gfgf\" (UniqueName: \"kubernetes.io/projected/9bc5eef8-d251-408c-9a91-e2d0f9ee9799-kube-api-access-7gfgf\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:16:51.085848 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:51.085807    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn" event={"ID":"9bc5eef8-d251-408c-9a91-e2d0f9ee9799","Type":"ContainerDied","Data":"d120e3ed0920bf444344a2223f9973fa8a17db8430660e2c5d93c019b0fed102"}
Jul 09 12:16:51.085993 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:51.085866    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d120e3ed0920bf444344a2223f9973fa8a17db8430660e2c5d93c019b0fed102"
Jul 09 12:16:51.085993 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:51.085828    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540v5sfn"
Jul 09 12:16:54.283889 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.283838    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bb8r4"]
Jul 09 12:16:54.284350 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.284294    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bc5eef8-d251-408c-9a91-e2d0f9ee9799" containerName="util"
Jul 09 12:16:54.284350 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.284311    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bc5eef8-d251-408c-9a91-e2d0f9ee9799" containerName="util"
Jul 09 12:16:54.284350 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.284330    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bc5eef8-d251-408c-9a91-e2d0f9ee9799" containerName="extract"
Jul 09 12:16:54.284350 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.284338    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bc5eef8-d251-408c-9a91-e2d0f9ee9799" containerName="extract"
Jul 09 12:16:54.284555 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.284355    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bc5eef8-d251-408c-9a91-e2d0f9ee9799" containerName="pull"
Jul 09 12:16:54.284555 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.284364    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bc5eef8-d251-408c-9a91-e2d0f9ee9799" containerName="pull"
Jul 09 12:16:54.284555 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.284440    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bc5eef8-d251-408c-9a91-e2d0f9ee9799" containerName="extract"
Jul 09 12:16:54.340849 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.340807    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bb8r4"]
Jul 09 12:16:54.340989 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.340935    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bb8r4"
Jul 09 12:16:54.343729 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.343686    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 09 12:16:54.343729 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.343727    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 09 12:16:54.343928 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.343775    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-vzjt6\""
Jul 09 12:16:54.483938 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.483897    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9c761c6-29a3-4394-b2a7-c0dc8ba418c4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bb8r4\" (UID: \"c9c761c6-29a3-4394-b2a7-c0dc8ba418c4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bb8r4"
Jul 09 12:16:54.484106 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.484017    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hpnn\" (UniqueName: \"kubernetes.io/projected/c9c761c6-29a3-4394-b2a7-c0dc8ba418c4-kube-api-access-9hpnn\") pod \"cert-manager-operator-controller-manager-55c444bf86-bb8r4\" (UID: \"c9c761c6-29a3-4394-b2a7-c0dc8ba418c4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bb8r4"
Jul 09 12:16:54.584875 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.584846    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9c761c6-29a3-4394-b2a7-c0dc8ba418c4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bb8r4\" (UID: \"c9c761c6-29a3-4394-b2a7-c0dc8ba418c4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bb8r4"
Jul 09 12:16:54.585042 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.584919    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hpnn\" (UniqueName: \"kubernetes.io/projected/c9c761c6-29a3-4394-b2a7-c0dc8ba418c4-kube-api-access-9hpnn\") pod \"cert-manager-operator-controller-manager-55c444bf86-bb8r4\" (UID: \"c9c761c6-29a3-4394-b2a7-c0dc8ba418c4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bb8r4"
Jul 09 12:16:54.585259 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.585240    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9c761c6-29a3-4394-b2a7-c0dc8ba418c4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bb8r4\" (UID: \"c9c761c6-29a3-4394-b2a7-c0dc8ba418c4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bb8r4"
Jul 09 12:16:54.613528 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.613503    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hpnn\" (UniqueName: \"kubernetes.io/projected/c9c761c6-29a3-4394-b2a7-c0dc8ba418c4-kube-api-access-9hpnn\") pod \"cert-manager-operator-controller-manager-55c444bf86-bb8r4\" (UID: \"c9c761c6-29a3-4394-b2a7-c0dc8ba418c4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bb8r4"
Jul 09 12:16:54.668387 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.668354    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bb8r4"
Jul 09 12:16:54.824157 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:54.824130    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bb8r4"]
Jul 09 12:16:54.826825 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:16:54.826801    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9c761c6_29a3_4394_b2a7_c0dc8ba418c4.slice/crio-f66d2a682e156c941811871e03788dc3e91125ef5b83d3037b510c605262c7ec WatchSource:0}: Error finding container f66d2a682e156c941811871e03788dc3e91125ef5b83d3037b510c605262c7ec: Status 404 returned error can't find the container with id f66d2a682e156c941811871e03788dc3e91125ef5b83d3037b510c605262c7ec
Jul 09 12:16:55.102089 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:55.102053    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bb8r4" event={"ID":"c9c761c6-29a3-4394-b2a7-c0dc8ba418c4","Type":"ContainerStarted","Data":"f66d2a682e156c941811871e03788dc3e91125ef5b83d3037b510c605262c7ec"}
Jul 09 12:16:58.117218 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:58.117175    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bb8r4" event={"ID":"c9c761c6-29a3-4394-b2a7-c0dc8ba418c4","Type":"ContainerStarted","Data":"9cf06e4ca992daf05069b2963d54e0bfb5dd68ba059bfaca0d1b410fc30d81a1"}
Jul 09 12:16:58.141941 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:58.141875    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bb8r4" podStartSLOduration=1.8709992629999999 podStartE2EDuration="4.141852679s" totalImagesPullingTime="2.270853416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:16:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:16:58.139127363 +0000 UTC m=+890.816656421" watchObservedRunningTime="2026-07-09 12:16:58.141852679 +0000 UTC m=+890.819381737"
Jul 09 12:16:59.086652 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.086567    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-v4n8m"]
Jul 09 12:16:59.088986 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.088968    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-v4n8m"
Jul 09 12:16:59.091818 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.091798    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 09 12:16:59.091921 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.091837    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 09 12:16:59.093029 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.093007    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-2dfrd\""
Jul 09 12:16:59.105927 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.105905    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-v4n8m"]
Jul 09 12:16:59.220189 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.220156    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"]
Jul 09 12:16:59.222591 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.222575    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"
Jul 09 12:16:59.225145 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.225123    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 12:16:59.225250 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.225127    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 12:16:59.225325 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.225244    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v46hf\""
Jul 09 12:16:59.225325 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.225310    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7wps\" (UniqueName: \"kubernetes.io/projected/453e80c7-4ee6-4b22-846f-80f451692d86-kube-api-access-m7wps\") pod \"cert-manager-webhook-6486ffd997-v4n8m\" (UID: \"453e80c7-4ee6-4b22-846f-80f451692d86\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v4n8m"
Jul 09 12:16:59.225434 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.225373    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/453e80c7-4ee6-4b22-846f-80f451692d86-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-v4n8m\" (UID: \"453e80c7-4ee6-4b22-846f-80f451692d86\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v4n8m"
Jul 09 12:16:59.233598 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.233579    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"]
Jul 09 12:16:59.326387 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.326353    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgfn8\" (UniqueName: \"kubernetes.io/projected/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-kube-api-access-bgfn8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2\" (UID: \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"
Jul 09 12:16:59.326551 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.326401    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7wps\" (UniqueName: \"kubernetes.io/projected/453e80c7-4ee6-4b22-846f-80f451692d86-kube-api-access-m7wps\") pod \"cert-manager-webhook-6486ffd997-v4n8m\" (UID: \"453e80c7-4ee6-4b22-846f-80f451692d86\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v4n8m"
Jul 09 12:16:59.326551 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.326454    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/453e80c7-4ee6-4b22-846f-80f451692d86-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-v4n8m\" (UID: \"453e80c7-4ee6-4b22-846f-80f451692d86\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v4n8m"
Jul 09 12:16:59.326551 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.326499    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2\" (UID: \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"
Jul 09 12:16:59.326551 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.326523    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2\" (UID: \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"
Jul 09 12:16:59.337887 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.337837    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/453e80c7-4ee6-4b22-846f-80f451692d86-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-v4n8m\" (UID: \"453e80c7-4ee6-4b22-846f-80f451692d86\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v4n8m"
Jul 09 12:16:59.340792 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.340768    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7wps\" (UniqueName: \"kubernetes.io/projected/453e80c7-4ee6-4b22-846f-80f451692d86-kube-api-access-m7wps\") pod \"cert-manager-webhook-6486ffd997-v4n8m\" (UID: \"453e80c7-4ee6-4b22-846f-80f451692d86\") " pod="cert-manager/cert-manager-webhook-6486ffd997-v4n8m"
Jul 09 12:16:59.398160 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.398128    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-v4n8m"
Jul 09 12:16:59.427513 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.427485    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2\" (UID: \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"
Jul 09 12:16:59.427654 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.427519    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2\" (UID: \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"
Jul 09 12:16:59.427654 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.427559    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgfn8\" (UniqueName: \"kubernetes.io/projected/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-kube-api-access-bgfn8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2\" (UID: \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"
Jul 09 12:16:59.427873 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.427846    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2\" (UID: \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"
Jul 09 12:16:59.427985 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.427890    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2\" (UID: \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"
Jul 09 12:16:59.436962 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.436938    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgfn8\" (UniqueName: \"kubernetes.io/projected/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-kube-api-access-bgfn8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2\" (UID: \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"
Jul 09 12:16:59.531462 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.531428    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"
Jul 09 12:16:59.538314 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.538293    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-v4n8m"]
Jul 09 12:16:59.540828 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:16:59.540787    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod453e80c7_4ee6_4b22_846f_80f451692d86.slice/crio-09d78949d3c20dd61279646b439bd3f065b2c519c12aa937d5663508ea74ad3b WatchSource:0}: Error finding container 09d78949d3c20dd61279646b439bd3f065b2c519c12aa937d5663508ea74ad3b: Status 404 returned error can't find the container with id 09d78949d3c20dd61279646b439bd3f065b2c519c12aa937d5663508ea74ad3b
Jul 09 12:16:59.668512 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:16:59.668486    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"]
Jul 09 12:16:59.669984 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:16:59.669962    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6ae69f2_8eba_4004_8387_5c5813a8f8f3.slice/crio-022caa8fffa63c43ce0f191eca9108c0615c6347a0ffd54e60077ba1f69fff01 WatchSource:0}: Error finding container 022caa8fffa63c43ce0f191eca9108c0615c6347a0ffd54e60077ba1f69fff01: Status 404 returned error can't find the container with id 022caa8fffa63c43ce0f191eca9108c0615c6347a0ffd54e60077ba1f69fff01
Jul 09 12:17:00.132369 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:00.132317    2565 generic.go:356] "Generic (PLEG): container finished" podID="a6ae69f2-8eba-4004-8387-5c5813a8f8f3" containerID="839798edd1ea9ccd513f4bd55a8ef0a6a094daa86ba87f7d346771581302b847" exitCode=0
Jul 09 12:17:00.132576 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:00.132371    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2" event={"ID":"a6ae69f2-8eba-4004-8387-5c5813a8f8f3","Type":"ContainerDied","Data":"839798edd1ea9ccd513f4bd55a8ef0a6a094daa86ba87f7d346771581302b847"}
Jul 09 12:17:00.132576 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:00.132422    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2" event={"ID":"a6ae69f2-8eba-4004-8387-5c5813a8f8f3","Type":"ContainerStarted","Data":"022caa8fffa63c43ce0f191eca9108c0615c6347a0ffd54e60077ba1f69fff01"}
Jul 09 12:17:00.133585 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:00.133564    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-v4n8m" event={"ID":"453e80c7-4ee6-4b22-846f-80f451692d86","Type":"ContainerStarted","Data":"09d78949d3c20dd61279646b439bd3f065b2c519c12aa937d5663508ea74ad3b"}
Jul 09 12:17:04.155420 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:04.155385    2565 generic.go:356] "Generic (PLEG): container finished" podID="a6ae69f2-8eba-4004-8387-5c5813a8f8f3" containerID="af654cb9b5ca92e6d9312e423da90acdf759e0974bfec1c480eb238185940427" exitCode=0
Jul 09 12:17:04.155917 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:04.155499    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2" event={"ID":"a6ae69f2-8eba-4004-8387-5c5813a8f8f3","Type":"ContainerDied","Data":"af654cb9b5ca92e6d9312e423da90acdf759e0974bfec1c480eb238185940427"}
Jul 09 12:17:04.157061 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:04.157030    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-v4n8m" event={"ID":"453e80c7-4ee6-4b22-846f-80f451692d86","Type":"ContainerStarted","Data":"2ae43f71b7e45c8670b7d8df0ce838735b08f0851602aec9ee7443f5064b7a93"}
Jul 09 12:17:04.157182 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:04.157084    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-v4n8m"
Jul 09 12:17:04.201571 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:04.201522    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-v4n8m" podStartSLOduration=1.234335533 podStartE2EDuration="5.201506235s" totalImagesPullingTime="3.967170702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:17:04.19959228 +0000 UTC m=+896.877121342" watchObservedRunningTime="2026-07-09 12:17:04.201506235 +0000 UTC m=+896.879035287"
Jul 09 12:17:05.163137 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:05.163108    2565 generic.go:356] "Generic (PLEG): container finished" podID="a6ae69f2-8eba-4004-8387-5c5813a8f8f3" containerID="eb2a6172c572ef91169ac5a0d8c29ba34381bfb689ac6c14daea88edbfaaed3d" exitCode=0
Jul 09 12:17:05.163533 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:05.163169    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2" event={"ID":"a6ae69f2-8eba-4004-8387-5c5813a8f8f3","Type":"ContainerDied","Data":"eb2a6172c572ef91169ac5a0d8c29ba34381bfb689ac6c14daea88edbfaaed3d"}
Jul 09 12:17:06.291810 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:06.291785    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"
Jul 09 12:17:06.386465 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:06.386422    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgfn8\" (UniqueName: \"kubernetes.io/projected/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-kube-api-access-bgfn8\") pod \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\" (UID: \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\") "
Jul 09 12:17:06.386615 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:06.386513    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-util\") pod \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\" (UID: \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\") "
Jul 09 12:17:06.386615 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:06.386554    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-bundle\") pod \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\" (UID: \"a6ae69f2-8eba-4004-8387-5c5813a8f8f3\") "
Jul 09 12:17:06.387020 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:06.386993    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-bundle" (OuterVolumeSpecName: "bundle") pod "a6ae69f2-8eba-4004-8387-5c5813a8f8f3" (UID: "a6ae69f2-8eba-4004-8387-5c5813a8f8f3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:17:06.388719 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:06.388675    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-kube-api-access-bgfn8" (OuterVolumeSpecName: "kube-api-access-bgfn8") pod "a6ae69f2-8eba-4004-8387-5c5813a8f8f3" (UID: "a6ae69f2-8eba-4004-8387-5c5813a8f8f3"). InnerVolumeSpecName "kube-api-access-bgfn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:17:06.390873 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:06.390854    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-util" (OuterVolumeSpecName: "util") pod "a6ae69f2-8eba-4004-8387-5c5813a8f8f3" (UID: "a6ae69f2-8eba-4004-8387-5c5813a8f8f3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:17:06.487297 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:06.487215    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-util\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:17:06.487297 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:06.487241    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-bundle\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:17:06.487297 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:06.487250    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgfn8\" (UniqueName: \"kubernetes.io/projected/a6ae69f2-8eba-4004-8387-5c5813a8f8f3-kube-api-access-bgfn8\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:17:07.173383 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:07.173355    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2"
Jul 09 12:17:07.173383 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:07.173366    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzkhl2" event={"ID":"a6ae69f2-8eba-4004-8387-5c5813a8f8f3","Type":"ContainerDied","Data":"022caa8fffa63c43ce0f191eca9108c0615c6347a0ffd54e60077ba1f69fff01"}
Jul 09 12:17:07.173633 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:07.173408    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="022caa8fffa63c43ce0f191eca9108c0615c6347a0ffd54e60077ba1f69fff01"
Jul 09 12:17:07.834821 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:07.834791    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:17:07.835270 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:07.834971    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:17:10.165013 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:10.164986    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-v4n8m"
Jul 09 12:17:18.612646 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.612606    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"]
Jul 09 12:17:18.613051 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.612912    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6ae69f2-8eba-4004-8387-5c5813a8f8f3" containerName="pull"
Jul 09 12:17:18.613051 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.612924    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6ae69f2-8eba-4004-8387-5c5813a8f8f3" containerName="pull"
Jul 09 12:17:18.613051 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.612937    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6ae69f2-8eba-4004-8387-5c5813a8f8f3" containerName="util"
Jul 09 12:17:18.613051 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.612942    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6ae69f2-8eba-4004-8387-5c5813a8f8f3" containerName="util"
Jul 09 12:17:18.613051 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.612954    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6ae69f2-8eba-4004-8387-5c5813a8f8f3" containerName="extract"
Jul 09 12:17:18.613051 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.612960    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6ae69f2-8eba-4004-8387-5c5813a8f8f3" containerName="extract"
Jul 09 12:17:18.613051 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.613014    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6ae69f2-8eba-4004-8387-5c5813a8f8f3" containerName="extract"
Jul 09 12:17:18.634837 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.634804    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"]
Jul 09 12:17:18.635003 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.634898    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"
Jul 09 12:17:18.635003 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.634948    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ng8sf"]
Jul 09 12:17:18.637582 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.637560    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v46hf\""
Jul 09 12:17:18.637732 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.637612    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 12:17:18.638685 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.638662    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 12:17:18.649000 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.648977    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ng8sf"
Jul 09 12:17:18.649089 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.648983    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ng8sf"]
Jul 09 12:17:18.651615 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.651585    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ssd5g\""
Jul 09 12:17:18.678411 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.678389    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/714b96b5-5408-4e16-a8ea-e05603cbe9a2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd\" (UID: \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"
Jul 09 12:17:18.678499 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.678429    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47aef622-fc49-47c6-8fa0-4a604d1d8b8e-bound-sa-token\") pod \"cert-manager-55b4d954d5-ng8sf\" (UID: \"47aef622-fc49-47c6-8fa0-4a604d1d8b8e\") " pod="cert-manager/cert-manager-55b4d954d5-ng8sf"
Jul 09 12:17:18.678499 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.678447    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/714b96b5-5408-4e16-a8ea-e05603cbe9a2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd\" (UID: \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"
Jul 09 12:17:18.678499 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.678470    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gxdn\" (UniqueName: \"kubernetes.io/projected/47aef622-fc49-47c6-8fa0-4a604d1d8b8e-kube-api-access-6gxdn\") pod \"cert-manager-55b4d954d5-ng8sf\" (UID: \"47aef622-fc49-47c6-8fa0-4a604d1d8b8e\") " pod="cert-manager/cert-manager-55b4d954d5-ng8sf"
Jul 09 12:17:18.678602 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.678513    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttb84\" (UniqueName: \"kubernetes.io/projected/714b96b5-5408-4e16-a8ea-e05603cbe9a2-kube-api-access-ttb84\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd\" (UID: \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"
Jul 09 12:17:18.779760 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.779738    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47aef622-fc49-47c6-8fa0-4a604d1d8b8e-bound-sa-token\") pod \"cert-manager-55b4d954d5-ng8sf\" (UID: \"47aef622-fc49-47c6-8fa0-4a604d1d8b8e\") " pod="cert-manager/cert-manager-55b4d954d5-ng8sf"
Jul 09 12:17:18.779867 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.779767    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/714b96b5-5408-4e16-a8ea-e05603cbe9a2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd\" (UID: \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"
Jul 09 12:17:18.779867 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.779789    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gxdn\" (UniqueName: \"kubernetes.io/projected/47aef622-fc49-47c6-8fa0-4a604d1d8b8e-kube-api-access-6gxdn\") pod \"cert-manager-55b4d954d5-ng8sf\" (UID: \"47aef622-fc49-47c6-8fa0-4a604d1d8b8e\") " pod="cert-manager/cert-manager-55b4d954d5-ng8sf"
Jul 09 12:17:18.779944 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.779860    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttb84\" (UniqueName: \"kubernetes.io/projected/714b96b5-5408-4e16-a8ea-e05603cbe9a2-kube-api-access-ttb84\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd\" (UID: \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"
Jul 09 12:17:18.779978 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.779946    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/714b96b5-5408-4e16-a8ea-e05603cbe9a2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd\" (UID: \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"
Jul 09 12:17:18.780185 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.780165    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/714b96b5-5408-4e16-a8ea-e05603cbe9a2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd\" (UID: \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"
Jul 09 12:17:18.780236 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.780223    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/714b96b5-5408-4e16-a8ea-e05603cbe9a2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd\" (UID: \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"
Jul 09 12:17:18.788021 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.788001    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47aef622-fc49-47c6-8fa0-4a604d1d8b8e-bound-sa-token\") pod \"cert-manager-55b4d954d5-ng8sf\" (UID: \"47aef622-fc49-47c6-8fa0-4a604d1d8b8e\") " pod="cert-manager/cert-manager-55b4d954d5-ng8sf"
Jul 09 12:17:18.788136 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.788055    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttb84\" (UniqueName: \"kubernetes.io/projected/714b96b5-5408-4e16-a8ea-e05603cbe9a2-kube-api-access-ttb84\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd\" (UID: \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"
Jul 09 12:17:18.788136 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.788068    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gxdn\" (UniqueName: \"kubernetes.io/projected/47aef622-fc49-47c6-8fa0-4a604d1d8b8e-kube-api-access-6gxdn\") pod \"cert-manager-55b4d954d5-ng8sf\" (UID: \"47aef622-fc49-47c6-8fa0-4a604d1d8b8e\") " pod="cert-manager/cert-manager-55b4d954d5-ng8sf"
Jul 09 12:17:18.943853 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.943781    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"
Jul 09 12:17:18.957963 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:18.957945    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ng8sf"
Jul 09 12:17:19.101290 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:19.101257    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"]
Jul 09 12:17:19.104635 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:17:19.104600    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod714b96b5_5408_4e16_a8ea_e05603cbe9a2.slice/crio-0e3845b817afa782dece093a8d615170a9e43f7844597a5fbeb7914199c83d8a WatchSource:0}: Error finding container 0e3845b817afa782dece093a8d615170a9e43f7844597a5fbeb7914199c83d8a: Status 404 returned error can't find the container with id 0e3845b817afa782dece093a8d615170a9e43f7844597a5fbeb7914199c83d8a
Jul 09 12:17:19.135466 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:19.135431    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ng8sf"]
Jul 09 12:17:19.138968 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:17:19.138940    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47aef622_fc49_47c6_8fa0_4a604d1d8b8e.slice/crio-c72d5db7ff1d49099935222d2b36f3b9615d2ed2f0460433c985b122d8f90457 WatchSource:0}: Error finding container c72d5db7ff1d49099935222d2b36f3b9615d2ed2f0460433c985b122d8f90457: Status 404 returned error can't find the container with id c72d5db7ff1d49099935222d2b36f3b9615d2ed2f0460433c985b122d8f90457
Jul 09 12:17:19.231974 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:19.231934    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ng8sf" event={"ID":"47aef622-fc49-47c6-8fa0-4a604d1d8b8e","Type":"ContainerStarted","Data":"c72d5db7ff1d49099935222d2b36f3b9615d2ed2f0460433c985b122d8f90457"}
Jul 09 12:17:19.233693 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:19.233656    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd" event={"ID":"714b96b5-5408-4e16-a8ea-e05603cbe9a2","Type":"ContainerStarted","Data":"269d13e2c6c53a6d820db73405343796405fed4687c3f7c2780acca5b269a905"}
Jul 09 12:17:19.233870 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:19.233798    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd" event={"ID":"714b96b5-5408-4e16-a8ea-e05603cbe9a2","Type":"ContainerStarted","Data":"0e3845b817afa782dece093a8d615170a9e43f7844597a5fbeb7914199c83d8a"}
Jul 09 12:17:20.239916 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:20.239876    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ng8sf" event={"ID":"47aef622-fc49-47c6-8fa0-4a604d1d8b8e","Type":"ContainerStarted","Data":"87fe444250e4ea64d46598c36f9bfc272484e22b562d6d08061bb8a6c16ade02"}
Jul 09 12:17:20.241324 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:20.241300    2565 generic.go:356] "Generic (PLEG): container finished" podID="714b96b5-5408-4e16-a8ea-e05603cbe9a2" containerID="269d13e2c6c53a6d820db73405343796405fed4687c3f7c2780acca5b269a905" exitCode=0
Jul 09 12:17:20.241433 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:20.241382    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd" event={"ID":"714b96b5-5408-4e16-a8ea-e05603cbe9a2","Type":"ContainerDied","Data":"269d13e2c6c53a6d820db73405343796405fed4687c3f7c2780acca5b269a905"}
Jul 09 12:17:20.260499 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:20.260451    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-ng8sf" podStartSLOduration=2.260433664 podStartE2EDuration="2.260433664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:17:20.259324273 +0000 UTC m=+912.936853331" watchObservedRunningTime="2026-07-09 12:17:20.260433664 +0000 UTC m=+912.937962721"
Jul 09 12:17:22.252018 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:22.251984    2565 generic.go:356] "Generic (PLEG): container finished" podID="714b96b5-5408-4e16-a8ea-e05603cbe9a2" containerID="cb99c9aefbc6f9404951636c535228b2387f28c89e91ea9d001c8bf1adce5294" exitCode=0
Jul 09 12:17:22.252526 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:22.252048    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd" event={"ID":"714b96b5-5408-4e16-a8ea-e05603cbe9a2","Type":"ContainerDied","Data":"cb99c9aefbc6f9404951636c535228b2387f28c89e91ea9d001c8bf1adce5294"}
Jul 09 12:17:23.258320 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:23.258283    2565 generic.go:356] "Generic (PLEG): container finished" podID="714b96b5-5408-4e16-a8ea-e05603cbe9a2" containerID="b2820c566ed290c438ceddb12e9e629b3666f31c357938d2152d304a995239eb" exitCode=0
Jul 09 12:17:23.258679 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:23.258366    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd" event={"ID":"714b96b5-5408-4e16-a8ea-e05603cbe9a2","Type":"ContainerDied","Data":"b2820c566ed290c438ceddb12e9e629b3666f31c357938d2152d304a995239eb"}
Jul 09 12:17:24.392376 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:24.392353    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"
Jul 09 12:17:24.425125 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:24.425088    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/714b96b5-5408-4e16-a8ea-e05603cbe9a2-bundle\") pod \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\" (UID: \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\") "
Jul 09 12:17:24.425284 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:24.425134    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/714b96b5-5408-4e16-a8ea-e05603cbe9a2-util\") pod \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\" (UID: \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\") "
Jul 09 12:17:24.425284 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:24.425166    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttb84\" (UniqueName: \"kubernetes.io/projected/714b96b5-5408-4e16-a8ea-e05603cbe9a2-kube-api-access-ttb84\") pod \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\" (UID: \"714b96b5-5408-4e16-a8ea-e05603cbe9a2\") "
Jul 09 12:17:24.425820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:24.425796    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/714b96b5-5408-4e16-a8ea-e05603cbe9a2-bundle" (OuterVolumeSpecName: "bundle") pod "714b96b5-5408-4e16-a8ea-e05603cbe9a2" (UID: "714b96b5-5408-4e16-a8ea-e05603cbe9a2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:17:24.427269 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:24.427241    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/714b96b5-5408-4e16-a8ea-e05603cbe9a2-kube-api-access-ttb84" (OuterVolumeSpecName: "kube-api-access-ttb84") pod "714b96b5-5408-4e16-a8ea-e05603cbe9a2" (UID: "714b96b5-5408-4e16-a8ea-e05603cbe9a2"). InnerVolumeSpecName "kube-api-access-ttb84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:17:24.500005 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:24.499966    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/714b96b5-5408-4e16-a8ea-e05603cbe9a2-util" (OuterVolumeSpecName: "util") pod "714b96b5-5408-4e16-a8ea-e05603cbe9a2" (UID: "714b96b5-5408-4e16-a8ea-e05603cbe9a2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:17:24.526299 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:24.526252    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttb84\" (UniqueName: \"kubernetes.io/projected/714b96b5-5408-4e16-a8ea-e05603cbe9a2-kube-api-access-ttb84\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:17:24.526299 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:24.526273    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/714b96b5-5408-4e16-a8ea-e05603cbe9a2-bundle\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:17:24.526299 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:24.526283    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/714b96b5-5408-4e16-a8ea-e05603cbe9a2-util\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:17:25.274008 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:25.273983    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd"
Jul 09 12:17:25.274154 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:25.273983    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5hwzd" event={"ID":"714b96b5-5408-4e16-a8ea-e05603cbe9a2","Type":"ContainerDied","Data":"0e3845b817afa782dece093a8d615170a9e43f7844597a5fbeb7914199c83d8a"}
Jul 09 12:17:25.274154 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:25.274096    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e3845b817afa782dece093a8d615170a9e43f7844597a5fbeb7914199c83d8a"
Jul 09 12:17:35.769485 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.769450    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"]
Jul 09 12:17:35.770078 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.769796    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="714b96b5-5408-4e16-a8ea-e05603cbe9a2" containerName="util"
Jul 09 12:17:35.770078 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.769808    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="714b96b5-5408-4e16-a8ea-e05603cbe9a2" containerName="util"
Jul 09 12:17:35.770078 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.769822    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="714b96b5-5408-4e16-a8ea-e05603cbe9a2" containerName="extract"
Jul 09 12:17:35.770078 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.769827    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="714b96b5-5408-4e16-a8ea-e05603cbe9a2" containerName="extract"
Jul 09 12:17:35.770078 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.769841    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="714b96b5-5408-4e16-a8ea-e05603cbe9a2" containerName="pull"
Jul 09 12:17:35.770078 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.769846    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="714b96b5-5408-4e16-a8ea-e05603cbe9a2" containerName="pull"
Jul 09 12:17:35.770078 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.769901    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="714b96b5-5408-4e16-a8ea-e05603cbe9a2" containerName="extract"
Jul 09 12:17:35.773142 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.773120    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.775907 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.775884    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 09 12:17:35.776032 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.775920    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 09 12:17:35.776346 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.776324    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 09 12:17:35.776346 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.776338    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fzkqb\""
Jul 09 12:17:35.776487 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.776355    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 09 12:17:35.788948 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.788923    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"]
Jul 09 12:17:35.819395 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.819357    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpx6g\" (UniqueName: \"kubernetes.io/projected/4a0bbd67-ef5e-47c1-8f29-cde9280de159-kube-api-access-vpx6g\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.819658 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.819638    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a0bbd67-ef5e-47c1-8f29-cde9280de159-tmp\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.819820 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.819804    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a0bbd67-ef5e-47c1-8f29-cde9280de159-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.819927 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.819914    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a0bbd67-ef5e-47c1-8f29-cde9280de159-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.820055 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.820041    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a0bbd67-ef5e-47c1-8f29-cde9280de159-manifests\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.920787 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.920736    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a0bbd67-ef5e-47c1-8f29-cde9280de159-manifests\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.920966 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.920837    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpx6g\" (UniqueName: \"kubernetes.io/projected/4a0bbd67-ef5e-47c1-8f29-cde9280de159-kube-api-access-vpx6g\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.920966 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.920864    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a0bbd67-ef5e-47c1-8f29-cde9280de159-tmp\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.920966 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.920887    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a0bbd67-ef5e-47c1-8f29-cde9280de159-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.920966 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.920932    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a0bbd67-ef5e-47c1-8f29-cde9280de159-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.921169 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.921108    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4a0bbd67-ef5e-47c1-8f29-cde9280de159-manifests\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.923400 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.923369    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a0bbd67-ef5e-47c1-8f29-cde9280de159-tmp\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.923520 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.923460    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a0bbd67-ef5e-47c1-8f29-cde9280de159-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.923562 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.923549    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4a0bbd67-ef5e-47c1-8f29-cde9280de159-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:35.929124 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:35.929099    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpx6g\" (UniqueName: \"kubernetes.io/projected/4a0bbd67-ef5e-47c1-8f29-cde9280de159-kube-api-access-vpx6g\") pod \"opendatahub-operator-controller-manager-7bd7989cf5-5d2r2\" (UID: \"4a0bbd67-ef5e-47c1-8f29-cde9280de159\") " pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:36.085290 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:36.085264    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:36.240842 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:36.240817    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"]
Jul 09 12:17:36.242858 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:17:36.242834    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a0bbd67_ef5e_47c1_8f29_cde9280de159.slice/crio-a26b90b17ace1e4c0faba7ae5aacce07700776047687c7023b4a35be4bb768a1 WatchSource:0}: Error finding container a26b90b17ace1e4c0faba7ae5aacce07700776047687c7023b4a35be4bb768a1: Status 404 returned error can't find the container with id a26b90b17ace1e4c0faba7ae5aacce07700776047687c7023b4a35be4bb768a1
Jul 09 12:17:36.323314 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:36.323273    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2" event={"ID":"4a0bbd67-ef5e-47c1-8f29-cde9280de159","Type":"ContainerStarted","Data":"a26b90b17ace1e4c0faba7ae5aacce07700776047687c7023b4a35be4bb768a1"}
Jul 09 12:17:40.344674 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:40.344638    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2" event={"ID":"4a0bbd67-ef5e-47c1-8f29-cde9280de159","Type":"ContainerStarted","Data":"2a40b0f2c7ef55798339758ab49c2b0ffe805c6ae07605e0c04868815ece310c"}
Jul 09 12:17:41.351418 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:41.351390    2565 generic.go:356] "Generic (PLEG): container finished" podID="4a0bbd67-ef5e-47c1-8f29-cde9280de159" containerID="2a40b0f2c7ef55798339758ab49c2b0ffe805c6ae07605e0c04868815ece310c" exitCode=0
Jul 09 12:17:41.351794 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:41.351449    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2" event={"ID":"4a0bbd67-ef5e-47c1-8f29-cde9280de159","Type":"ContainerDied","Data":"2a40b0f2c7ef55798339758ab49c2b0ffe805c6ae07605e0c04868815ece310c"}
Jul 09 12:17:42.357883 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:42.357823    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2" event={"ID":"4a0bbd67-ef5e-47c1-8f29-cde9280de159","Type":"ContainerStarted","Data":"d8eb3eadfc138321c46b17558221b72669b94f0ed9275e14b937db427ec51827"}
Jul 09 12:17:42.358243 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:42.357985    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:17:42.384588 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:42.384540    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2" podStartSLOduration=2.394433897 podStartE2EDuration="7.38452471s" totalImagesPullingTime="3.90094689s" totalInitContainerRuntime="1.089143923s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:17:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:17:42.381756129 +0000 UTC m=+935.059285188" watchObservedRunningTime="2026-07-09 12:17:42.38452471 +0000 UTC m=+935.062053768"
Jul 09 12:17:53.365378 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:17:53.365347    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7bd7989cf5-5d2r2"
Jul 09 12:18:04.343132 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.343092    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"]
Jul 09 12:18:04.346970 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.346944    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.352585 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.352553    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 09 12:18:04.352585 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.352573    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-xb4ds\""
Jul 09 12:18:04.352957 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.352938    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 09 12:18:04.353172 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.353105    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 09 12:18:04.364224 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.364196    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"]
Jul 09 12:18:04.470755 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.470721    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.470941 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.470776    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.470941 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.470830    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44lxv\" (UniqueName: \"kubernetes.io/projected/01f8f734-881a-45f1-8034-f0d405903459-kube-api-access-44lxv\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.470941 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.470853    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.470941 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.470923    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01f8f734-881a-45f1-8034-f0d405903459-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.471161 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.470964    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01f8f734-881a-45f1-8034-f0d405903459-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.471161 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.470991    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01f8f734-881a-45f1-8034-f0d405903459-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.471161 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.471051    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.471161 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.471120    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.572330 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.572295    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01f8f734-881a-45f1-8034-f0d405903459-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.572488 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.572344    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.572488 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.572376    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.572488 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.572395    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.572488 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.572439    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.572488 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.572465    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44lxv\" (UniqueName: \"kubernetes.io/projected/01f8f734-881a-45f1-8034-f0d405903459-kube-api-access-44lxv\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.572488 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.572485    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.572835 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.572807    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01f8f734-881a-45f1-8034-f0d405903459-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.572890 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.572868    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01f8f734-881a-45f1-8034-f0d405903459-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.572946 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.572900    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.573060 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.573028    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.573195 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.573069    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.573195 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.573131    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.573487 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.573435    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01f8f734-881a-45f1-8034-f0d405903459-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.574915 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.574895    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.575307 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.575286    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01f8f734-881a-45f1-8034-f0d405903459-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.603283 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.603207    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44lxv\" (UniqueName: \"kubernetes.io/projected/01f8f734-881a-45f1-8034-f0d405903459-kube-api-access-44lxv\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.604499 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.604478    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"]
Jul 09 12:18:04.608668 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.608647    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.610548 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.610525    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01f8f734-881a-45f1-8034-f0d405903459-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-k845n\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.652795 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.652762    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"]
Jul 09 12:18:04.658814 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.658789    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:04.674013 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.673986    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/493c258a-fdf6-488c-9b7f-6a304b59b578-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.674137 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.674031    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.674137 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.674061    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.674137 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.674086    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs49t\" (UniqueName: \"kubernetes.io/projected/493c258a-fdf6-488c-9b7f-6a304b59b578-kube-api-access-rs49t\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.674137 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.674110    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.674298 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.674215    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.674298 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.674271    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.674298 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.674290    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/493c258a-fdf6-488c-9b7f-6a304b59b578-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.674444 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.674329    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/493c258a-fdf6-488c-9b7f-6a304b59b578-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.775353 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.775316    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/493c258a-fdf6-488c-9b7f-6a304b59b578-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.775520 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.775368    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.775520 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.775399    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.775520 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.775424    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs49t\" (UniqueName: \"kubernetes.io/projected/493c258a-fdf6-488c-9b7f-6a304b59b578-kube-api-access-rs49t\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.775520 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.775453    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.775520 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.775511    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.775839 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.775544    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.775839 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.775567    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/493c258a-fdf6-488c-9b7f-6a304b59b578-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.775839 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.775613    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/493c258a-fdf6-488c-9b7f-6a304b59b578-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.776007 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.775859    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.776007 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.775870    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.776271 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.776229    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.776445 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.776422    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.776737 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.776685    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/493c258a-fdf6-488c-9b7f-6a304b59b578-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.777979 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.777961    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/493c258a-fdf6-488c-9b7f-6a304b59b578-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.778044 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.777962    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/493c258a-fdf6-488c-9b7f-6a304b59b578-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.802880 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.802859    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/493c258a-fdf6-488c-9b7f-6a304b59b578-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.803810 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.803794    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs49t\" (UniqueName: \"kubernetes.io/projected/493c258a-fdf6-488c-9b7f-6a304b59b578-kube-api-access-rs49t\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4sgxjr\" (UID: \"493c258a-fdf6-488c-9b7f-6a304b59b578\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.824431 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.824406    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"]
Jul 09 12:18:04.826138 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:18:04.826114    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01f8f734_881a_45f1_8034_f0d405903459.slice/crio-f9db3733e356a74fca7c6a499328dccfd1dee9314a2be81463033a911dea69e3 WatchSource:0}: Error finding container f9db3733e356a74fca7c6a499328dccfd1dee9314a2be81463033a911dea69e3: Status 404 returned error can't find the container with id f9db3733e356a74fca7c6a499328dccfd1dee9314a2be81463033a911dea69e3
Jul 09 12:18:04.924817 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.924733    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:04.935732 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.935691    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"]
Jul 09 12:18:04.938974 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.938957    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:04.941805 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.941781    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 09 12:18:04.942096 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.941883    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 09 12:18:04.942188 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.942148    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 09 12:18:04.942355 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.942316    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 09 12:18:04.942832 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.942813    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-w7jbx\""
Jul 09 12:18:04.945576 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.945548    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 09 12:18:04.961308 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.961282    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"]
Jul 09 12:18:04.977306 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.977115    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/31e5831b-47b2-49b4-8a8c-b93f730edc3b-metrics-cert\") pod \"lws-controller-manager-849fbb9d-frqzs\" (UID: \"31e5831b-47b2-49b4-8a8c-b93f730edc3b\") " pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:04.977306 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.977159    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5wl4\" (UniqueName: \"kubernetes.io/projected/31e5831b-47b2-49b4-8a8c-b93f730edc3b-kube-api-access-t5wl4\") pod \"lws-controller-manager-849fbb9d-frqzs\" (UID: \"31e5831b-47b2-49b4-8a8c-b93f730edc3b\") " pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:04.977306 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.977233    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/31e5831b-47b2-49b4-8a8c-b93f730edc3b-manager-config\") pod \"lws-controller-manager-849fbb9d-frqzs\" (UID: \"31e5831b-47b2-49b4-8a8c-b93f730edc3b\") " pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:04.977480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:04.977363    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31e5831b-47b2-49b4-8a8c-b93f730edc3b-cert\") pod \"lws-controller-manager-849fbb9d-frqzs\" (UID: \"31e5831b-47b2-49b4-8a8c-b93f730edc3b\") " pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:05.069814 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:05.069788    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"]
Jul 09 12:18:05.071810 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:18:05.071784    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod493c258a_fdf6_488c_9b7f_6a304b59b578.slice/crio-94d3d32f32d13f3e2b9ef291555d8f1e6ac333f00f8716f5036eca85db511a5c WatchSource:0}: Error finding container 94d3d32f32d13f3e2b9ef291555d8f1e6ac333f00f8716f5036eca85db511a5c: Status 404 returned error can't find the container with id 94d3d32f32d13f3e2b9ef291555d8f1e6ac333f00f8716f5036eca85db511a5c
Jul 09 12:18:05.078611 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:05.078590    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31e5831b-47b2-49b4-8a8c-b93f730edc3b-cert\") pod \"lws-controller-manager-849fbb9d-frqzs\" (UID: \"31e5831b-47b2-49b4-8a8c-b93f730edc3b\") " pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:05.078691 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:05.078651    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/31e5831b-47b2-49b4-8a8c-b93f730edc3b-metrics-cert\") pod \"lws-controller-manager-849fbb9d-frqzs\" (UID: \"31e5831b-47b2-49b4-8a8c-b93f730edc3b\") " pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:05.078836 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:05.078818    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5wl4\" (UniqueName: \"kubernetes.io/projected/31e5831b-47b2-49b4-8a8c-b93f730edc3b-kube-api-access-t5wl4\") pod \"lws-controller-manager-849fbb9d-frqzs\" (UID: \"31e5831b-47b2-49b4-8a8c-b93f730edc3b\") " pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:05.078911 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:05.078893    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/31e5831b-47b2-49b4-8a8c-b93f730edc3b-manager-config\") pod \"lws-controller-manager-849fbb9d-frqzs\" (UID: \"31e5831b-47b2-49b4-8a8c-b93f730edc3b\") " pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:05.079558 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:05.079532    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/31e5831b-47b2-49b4-8a8c-b93f730edc3b-manager-config\") pod \"lws-controller-manager-849fbb9d-frqzs\" (UID: \"31e5831b-47b2-49b4-8a8c-b93f730edc3b\") " pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:05.081132 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:05.081110    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31e5831b-47b2-49b4-8a8c-b93f730edc3b-cert\") pod \"lws-controller-manager-849fbb9d-frqzs\" (UID: \"31e5831b-47b2-49b4-8a8c-b93f730edc3b\") " pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:05.081261 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:05.081171    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/31e5831b-47b2-49b4-8a8c-b93f730edc3b-metrics-cert\") pod \"lws-controller-manager-849fbb9d-frqzs\" (UID: \"31e5831b-47b2-49b4-8a8c-b93f730edc3b\") " pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:05.089052 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:05.089031    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5wl4\" (UniqueName: \"kubernetes.io/projected/31e5831b-47b2-49b4-8a8c-b93f730edc3b-kube-api-access-t5wl4\") pod \"lws-controller-manager-849fbb9d-frqzs\" (UID: \"31e5831b-47b2-49b4-8a8c-b93f730edc3b\") " pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:05.249431 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:05.249338    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:05.408796 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:18:05.408754    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e5831b_47b2_49b4_8a8c_b93f730edc3b.slice/crio-6c4406048631240e61fc45d80428a8ebaf5f140192a511d30a5a32211020193a WatchSource:0}: Error finding container 6c4406048631240e61fc45d80428a8ebaf5f140192a511d30a5a32211020193a: Status 404 returned error can't find the container with id 6c4406048631240e61fc45d80428a8ebaf5f140192a511d30a5a32211020193a
Jul 09 12:18:05.411324 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:05.411224    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"]
Jul 09 12:18:05.467267 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:05.467227    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n" event={"ID":"01f8f734-881a-45f1-8034-f0d405903459","Type":"ContainerStarted","Data":"f9db3733e356a74fca7c6a499328dccfd1dee9314a2be81463033a911dea69e3"}
Jul 09 12:18:05.468746 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:05.468715    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr" event={"ID":"493c258a-fdf6-488c-9b7f-6a304b59b578","Type":"ContainerStarted","Data":"94d3d32f32d13f3e2b9ef291555d8f1e6ac333f00f8716f5036eca85db511a5c"}
Jul 09 12:18:05.470043 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:05.470012    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs" event={"ID":"31e5831b-47b2-49b4-8a8c-b93f730edc3b","Type":"ContainerStarted","Data":"6c4406048631240e61fc45d80428a8ebaf5f140192a511d30a5a32211020193a"}
Jul 09 12:18:07.407013 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:07.405364    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"]
Jul 09 12:18:07.411761 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:07.411724    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"
Jul 09 12:18:07.415445 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:07.415422    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-rj8fm\""
Jul 09 12:18:07.415603 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:07.415575    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 09 12:18:07.432218 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:07.432168    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"]
Jul 09 12:18:07.506306 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:07.506267    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bc4j\" (UniqueName: \"kubernetes.io/projected/4e58dcac-ffcf-4e3f-a36f-83591970b8c4-kube-api-access-6bc4j\") pod \"kserve-controller-manager-d6f4b65b8-jbkjg\" (UID: \"4e58dcac-ffcf-4e3f-a36f-83591970b8c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"
Jul 09 12:18:07.506575 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:07.506356    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e58dcac-ffcf-4e3f-a36f-83591970b8c4-cert\") pod \"kserve-controller-manager-d6f4b65b8-jbkjg\" (UID: \"4e58dcac-ffcf-4e3f-a36f-83591970b8c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"
Jul 09 12:18:07.607811 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:07.607772    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bc4j\" (UniqueName: \"kubernetes.io/projected/4e58dcac-ffcf-4e3f-a36f-83591970b8c4-kube-api-access-6bc4j\") pod \"kserve-controller-manager-d6f4b65b8-jbkjg\" (UID: \"4e58dcac-ffcf-4e3f-a36f-83591970b8c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"
Jul 09 12:18:07.607811 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:07.607819    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e58dcac-ffcf-4e3f-a36f-83591970b8c4-cert\") pod \"kserve-controller-manager-d6f4b65b8-jbkjg\" (UID: \"4e58dcac-ffcf-4e3f-a36f-83591970b8c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"
Jul 09 12:18:07.608013 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:18:07.607932    2565 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 09 12:18:07.608013 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:18:07.608005    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e58dcac-ffcf-4e3f-a36f-83591970b8c4-cert podName:4e58dcac-ffcf-4e3f-a36f-83591970b8c4 nodeName:}" failed. No retries permitted until 2026-07-09 12:18:08.107982591 +0000 UTC m=+960.785511639 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4e58dcac-ffcf-4e3f-a36f-83591970b8c4-cert") pod "kserve-controller-manager-d6f4b65b8-jbkjg" (UID: "4e58dcac-ffcf-4e3f-a36f-83591970b8c4") : secret "kserve-webhook-server-cert" not found
Jul 09 12:18:07.620817 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:07.620784    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bc4j\" (UniqueName: \"kubernetes.io/projected/4e58dcac-ffcf-4e3f-a36f-83591970b8c4-kube-api-access-6bc4j\") pod \"kserve-controller-manager-d6f4b65b8-jbkjg\" (UID: \"4e58dcac-ffcf-4e3f-a36f-83591970b8c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"
Jul 09 12:18:08.052480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.052423    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:18:08.052659 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.052529    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:18:08.052659 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.052580    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:18:08.059617 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.059586    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:18:08.059729 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.059652    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:18:08.059729 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.059680    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:18:08.113484 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.113455    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e58dcac-ffcf-4e3f-a36f-83591970b8c4-cert\") pod \"kserve-controller-manager-d6f4b65b8-jbkjg\" (UID: \"4e58dcac-ffcf-4e3f-a36f-83591970b8c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"
Jul 09 12:18:08.113626 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:18:08.113591    2565 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 09 12:18:08.113808 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:18:08.113667    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e58dcac-ffcf-4e3f-a36f-83591970b8c4-cert podName:4e58dcac-ffcf-4e3f-a36f-83591970b8c4 nodeName:}" failed. No retries permitted until 2026-07-09 12:18:09.113646125 +0000 UTC m=+961.791175174 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4e58dcac-ffcf-4e3f-a36f-83591970b8c4-cert") pod "kserve-controller-manager-d6f4b65b8-jbkjg" (UID: "4e58dcac-ffcf-4e3f-a36f-83591970b8c4") : secret "kserve-webhook-server-cert" not found
Jul 09 12:18:08.487665 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.487629    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr" event={"ID":"493c258a-fdf6-488c-9b7f-6a304b59b578","Type":"ContainerStarted","Data":"d652c83c5472fb5d24583137fe278cafce45699c7bf1b59ed812b16d7e5ffe43"}
Jul 09 12:18:08.489319 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.489295    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs" event={"ID":"31e5831b-47b2-49b4-8a8c-b93f730edc3b","Type":"ContainerStarted","Data":"47e10e87634609310277248e555d2c5de751b14671c7baefe80ed3927ccc3997"}
Jul 09 12:18:08.489450 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.489393    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:08.490686 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.490668    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n" event={"ID":"01f8f734-881a-45f1-8034-f0d405903459","Type":"ContainerStarted","Data":"62f14df7d1dadeb9aee26b49f93a92153f11db0ee927da608f350f67a647d339"}
Jul 09 12:18:08.539095 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.539025    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr" podStartSLOduration=1.5535218739999999 podStartE2EDuration="4.53900792s" totalImagesPullingTime="2.985486046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:18:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:18:08.537256906 +0000 UTC m=+961.214785976" watchObservedRunningTime="2026-07-09 12:18:08.53900792 +0000 UTC m=+961.216536977"
Jul 09 12:18:08.600939 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.600883    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n" podStartSLOduration=1.3767928729999999 podStartE2EDuration="4.600864574s" totalImagesPullingTime="3.224071701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:18:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:18:08.599397703 +0000 UTC m=+961.276926806" watchObservedRunningTime="2026-07-09 12:18:08.600864574 +0000 UTC m=+961.278393633"
Jul 09 12:18:08.601935 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.601902    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs" podStartSLOduration=1.961728193 podStartE2EDuration="4.601891309s" totalImagesPullingTime="2.640163116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:18:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:18:08.571031309 +0000 UTC m=+961.248560366" watchObservedRunningTime="2026-07-09 12:18:08.601891309 +0000 UTC m=+961.279420377"
Jul 09 12:18:08.658910 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.658858    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:08.925182 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:08.925141    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:09.122799 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:09.122759    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e58dcac-ffcf-4e3f-a36f-83591970b8c4-cert\") pod \"kserve-controller-manager-d6f4b65b8-jbkjg\" (UID: \"4e58dcac-ffcf-4e3f-a36f-83591970b8c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"
Jul 09 12:18:09.125332 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:09.125312    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e58dcac-ffcf-4e3f-a36f-83591970b8c4-cert\") pod \"kserve-controller-manager-d6f4b65b8-jbkjg\" (UID: \"4e58dcac-ffcf-4e3f-a36f-83591970b8c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"
Jul 09 12:18:09.228447 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:09.228375    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-rj8fm\""
Jul 09 12:18:09.236559 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:09.236540    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"
Jul 09 12:18:09.374432 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:09.374383    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"]
Jul 09 12:18:09.376720 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:18:09.376670    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e58dcac_ffcf_4e3f_a36f_83591970b8c4.slice/crio-7e9221c83f44538b6c1bfba19d80db85e300a9b0b38fccdcf48f66e17f1477c6 WatchSource:0}: Error finding container 7e9221c83f44538b6c1bfba19d80db85e300a9b0b38fccdcf48f66e17f1477c6: Status 404 returned error can't find the container with id 7e9221c83f44538b6c1bfba19d80db85e300a9b0b38fccdcf48f66e17f1477c6
Jul 09 12:18:09.497004 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:09.496922    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg" event={"ID":"4e58dcac-ffcf-4e3f-a36f-83591970b8c4","Type":"ContainerStarted","Data":"7e9221c83f44538b6c1bfba19d80db85e300a9b0b38fccdcf48f66e17f1477c6"}
Jul 09 12:18:09.661015 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:09.660979    2565 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-k845n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Jul 09 12:18:09.661185 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:09.661034    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n" podUID="01f8f734-881a-45f1-8034-f0d405903459" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Jul 09 12:18:09.931040 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:09.931003    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:10.502769 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:10.502724    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:10.503674 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:10.503650    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4sgxjr"
Jul 09 12:18:10.590277 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:10.590242    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"]
Jul 09 12:18:10.660247 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:10.660208    2565 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-k845n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Jul 09 12:18:10.660432 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:10.660274    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n" podUID="01f8f734-881a-45f1-8034-f0d405903459" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Jul 09 12:18:11.507239 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:11.507200    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n" podUID="01f8f734-881a-45f1-8034-f0d405903459" containerName="istio-proxy" containerID="cri-o://62f14df7d1dadeb9aee26b49f93a92153f11db0ee927da608f350f67a647d339" gracePeriod=30
Jul 09 12:18:12.513495 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:12.513409    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg" event={"ID":"4e58dcac-ffcf-4e3f-a36f-83591970b8c4","Type":"ContainerStarted","Data":"de84a3ccfcb6963dcc6f17b9c2c9735cc07b2982c6d99f4cc9db975ffa0e9744"}
Jul 09 12:18:12.513891 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:12.513655    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"
Jul 09 12:18:12.533213 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:12.533164    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg" podStartSLOduration=2.69307672 podStartE2EDuration="5.533148574s" totalImagesPullingTime="2.840071854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:18:12.530932061 +0000 UTC m=+965.208461120" watchObservedRunningTime="2026-07-09 12:18:12.533148574 +0000 UTC m=+965.210677633"
Jul 09 12:18:16.755746 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.755715    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:16.894513 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.894476    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-workload-certs\") pod \"01f8f734-881a-45f1-8034-f0d405903459\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") "
Jul 09 12:18:16.894732 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.894527    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44lxv\" (UniqueName: \"kubernetes.io/projected/01f8f734-881a-45f1-8034-f0d405903459-kube-api-access-44lxv\") pod \"01f8f734-881a-45f1-8034-f0d405903459\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") "
Jul 09 12:18:16.894732 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.894557    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01f8f734-881a-45f1-8034-f0d405903459-istio-token\") pod \"01f8f734-881a-45f1-8034-f0d405903459\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") "
Jul 09 12:18:16.894732 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.894578    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-workload-socket\") pod \"01f8f734-881a-45f1-8034-f0d405903459\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") "
Jul 09 12:18:16.894732 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.894598    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01f8f734-881a-45f1-8034-f0d405903459-istiod-ca-cert\") pod \"01f8f734-881a-45f1-8034-f0d405903459\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") "
Jul 09 12:18:16.894732 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.894628    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-credential-socket\") pod \"01f8f734-881a-45f1-8034-f0d405903459\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") "
Jul 09 12:18:16.894732 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.894730    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-istio-envoy\") pod \"01f8f734-881a-45f1-8034-f0d405903459\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") "
Jul 09 12:18:16.895042 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.894766    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-istio-data\") pod \"01f8f734-881a-45f1-8034-f0d405903459\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") "
Jul 09 12:18:16.895042 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.894791    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01f8f734-881a-45f1-8034-f0d405903459-istio-podinfo\") pod \"01f8f734-881a-45f1-8034-f0d405903459\" (UID: \"01f8f734-881a-45f1-8034-f0d405903459\") "
Jul 09 12:18:16.895042 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.894835    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "01f8f734-881a-45f1-8034-f0d405903459" (UID: "01f8f734-881a-45f1-8034-f0d405903459"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:18:16.895042 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.894936    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "01f8f734-881a-45f1-8034-f0d405903459" (UID: "01f8f734-881a-45f1-8034-f0d405903459"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:18:16.895042 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.895017    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "01f8f734-881a-45f1-8034-f0d405903459" (UID: "01f8f734-881a-45f1-8034-f0d405903459"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:18:16.895042 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.895038    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01f8f734-881a-45f1-8034-f0d405903459-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "01f8f734-881a-45f1-8034-f0d405903459" (UID: "01f8f734-881a-45f1-8034-f0d405903459"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:18:16.895295 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.895116    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-workload-socket\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:18:16.895295 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.895136    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01f8f734-881a-45f1-8034-f0d405903459-istiod-ca-cert\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:18:16.895295 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.895145    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-istio-data" (OuterVolumeSpecName: "istio-data") pod "01f8f734-881a-45f1-8034-f0d405903459" (UID: "01f8f734-881a-45f1-8034-f0d405903459"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:18:16.895295 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.895150    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-credential-socket\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:18:16.895295 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.895187    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-workload-certs\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:18:16.897104 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.897081    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "01f8f734-881a-45f1-8034-f0d405903459" (UID: "01f8f734-881a-45f1-8034-f0d405903459"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:18:16.897257 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.897212    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01f8f734-881a-45f1-8034-f0d405903459-istio-token" (OuterVolumeSpecName: "istio-token") pod "01f8f734-881a-45f1-8034-f0d405903459" (UID: "01f8f734-881a-45f1-8034-f0d405903459"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:18:16.897470 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.897454    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/01f8f734-881a-45f1-8034-f0d405903459-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "01f8f734-881a-45f1-8034-f0d405903459" (UID: "01f8f734-881a-45f1-8034-f0d405903459"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 12:18:16.897512 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.897484    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01f8f734-881a-45f1-8034-f0d405903459-kube-api-access-44lxv" (OuterVolumeSpecName: "kube-api-access-44lxv") pod "01f8f734-881a-45f1-8034-f0d405903459" (UID: "01f8f734-881a-45f1-8034-f0d405903459"). InnerVolumeSpecName "kube-api-access-44lxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:18:16.995747 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.995689    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-istio-envoy\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:18:16.995747 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.995743    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01f8f734-881a-45f1-8034-f0d405903459-istio-data\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:18:16.995747 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.995754    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01f8f734-881a-45f1-8034-f0d405903459-istio-podinfo\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:18:16.995956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.995763    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44lxv\" (UniqueName: \"kubernetes.io/projected/01f8f734-881a-45f1-8034-f0d405903459-kube-api-access-44lxv\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:18:16.995956 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:16.995772    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01f8f734-881a-45f1-8034-f0d405903459-istio-token\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:18:17.539564 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:17.539528    2565 generic.go:356] "Generic (PLEG): container finished" podID="01f8f734-881a-45f1-8034-f0d405903459" containerID="62f14df7d1dadeb9aee26b49f93a92153f11db0ee927da608f350f67a647d339" exitCode=0
Jul 09 12:18:17.539764 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:17.539580    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"
Jul 09 12:18:17.539764 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:17.539613    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n" event={"ID":"01f8f734-881a-45f1-8034-f0d405903459","Type":"ContainerDied","Data":"62f14df7d1dadeb9aee26b49f93a92153f11db0ee927da608f350f67a647d339"}
Jul 09 12:18:17.539764 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:17.539668    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n" event={"ID":"01f8f734-881a-45f1-8034-f0d405903459","Type":"ContainerDied","Data":"f9db3733e356a74fca7c6a499328dccfd1dee9314a2be81463033a911dea69e3"}
Jul 09 12:18:17.539764 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:17.539691    2565 scope.go:117] "RemoveContainer" containerID="62f14df7d1dadeb9aee26b49f93a92153f11db0ee927da608f350f67a647d339"
Jul 09 12:18:17.549305 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:17.549287    2565 scope.go:117] "RemoveContainer" containerID="62f14df7d1dadeb9aee26b49f93a92153f11db0ee927da608f350f67a647d339"
Jul 09 12:18:17.549543 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:18:17.549522    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62f14df7d1dadeb9aee26b49f93a92153f11db0ee927da608f350f67a647d339\": container with ID starting with 62f14df7d1dadeb9aee26b49f93a92153f11db0ee927da608f350f67a647d339 not found: ID does not exist" containerID="62f14df7d1dadeb9aee26b49f93a92153f11db0ee927da608f350f67a647d339"
Jul 09 12:18:17.549597 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:17.549552    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62f14df7d1dadeb9aee26b49f93a92153f11db0ee927da608f350f67a647d339"} err="failed to get container status \"62f14df7d1dadeb9aee26b49f93a92153f11db0ee927da608f350f67a647d339\": rpc error: code = NotFound desc = could not find container \"62f14df7d1dadeb9aee26b49f93a92153f11db0ee927da608f350f67a647d339\": container with ID starting with 62f14df7d1dadeb9aee26b49f93a92153f11db0ee927da608f350f67a647d339 not found: ID does not exist"
Jul 09 12:18:17.561503 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:17.561482    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"]
Jul 09 12:18:17.566148 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:17.566119    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-k845n"]
Jul 09 12:18:17.893673 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:17.893640    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01f8f734-881a-45f1-8034-f0d405903459" path="/var/lib/kubelet/pods/01f8f734-881a-45f1-8034-f0d405903459/volumes"
Jul 09 12:18:19.499848 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:19.499816    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-849fbb9d-frqzs"
Jul 09 12:18:43.523145 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:18:43.523110    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jbkjg"
Jul 09 12:19:12.227999 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.227961    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"]
Jul 09 12:19:12.228415 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.228302    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01f8f734-881a-45f1-8034-f0d405903459" containerName="istio-proxy"
Jul 09 12:19:12.228415 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.228315    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="01f8f734-881a-45f1-8034-f0d405903459" containerName="istio-proxy"
Jul 09 12:19:12.228415 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.228380    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="01f8f734-881a-45f1-8034-f0d405903459" containerName="istio-proxy"
Jul 09 12:19:12.231291 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.231274    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"
Jul 09 12:19:12.233858 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.233836    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 12:19:12.234018 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.233912    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 12:19:12.234795 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.234778    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-p6dx7\""
Jul 09 12:19:12.239955 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.239931    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"]
Jul 09 12:19:12.279145 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.279110    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d0ebd1d-d802-492e-93d4-340a5574b3c9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw\" (UID: \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"
Jul 09 12:19:12.279364 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.279157    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsrxc\" (UniqueName: \"kubernetes.io/projected/9d0ebd1d-d802-492e-93d4-340a5574b3c9-kube-api-access-fsrxc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw\" (UID: \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"
Jul 09 12:19:12.279364 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.279304    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d0ebd1d-d802-492e-93d4-340a5574b3c9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw\" (UID: \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"
Jul 09 12:19:12.379902 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.379865    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d0ebd1d-d802-492e-93d4-340a5574b3c9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw\" (UID: \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"
Jul 09 12:19:12.380090 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.379920    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d0ebd1d-d802-492e-93d4-340a5574b3c9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw\" (UID: \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"
Jul 09 12:19:12.380090 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.379950    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsrxc\" (UniqueName: \"kubernetes.io/projected/9d0ebd1d-d802-492e-93d4-340a5574b3c9-kube-api-access-fsrxc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw\" (UID: \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"
Jul 09 12:19:12.380307 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.380285    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d0ebd1d-d802-492e-93d4-340a5574b3c9-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw\" (UID: \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"
Jul 09 12:19:12.380377 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.380328    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d0ebd1d-d802-492e-93d4-340a5574b3c9-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw\" (UID: \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"
Jul 09 12:19:12.389891 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.389858    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsrxc\" (UniqueName: \"kubernetes.io/projected/9d0ebd1d-d802-492e-93d4-340a5574b3c9-kube-api-access-fsrxc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw\" (UID: \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"
Jul 09 12:19:12.540903 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.540809    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"
Jul 09 12:19:12.627578 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.627387    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"]
Jul 09 12:19:12.631686 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.631624    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"
Jul 09 12:19:12.638988 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.638955    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"]
Jul 09 12:19:12.682012 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.681987    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"]
Jul 09 12:19:12.683015 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.682991    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x2pd\" (UniqueName: \"kubernetes.io/projected/6efa932f-daab-4dfa-98af-eb221866d49f-kube-api-access-2x2pd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5\" (UID: \"6efa932f-daab-4dfa-98af-eb221866d49f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"
Jul 09 12:19:12.683107 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.683033    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6efa932f-daab-4dfa-98af-eb221866d49f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5\" (UID: \"6efa932f-daab-4dfa-98af-eb221866d49f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"
Jul 09 12:19:12.683163 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.683116    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6efa932f-daab-4dfa-98af-eb221866d49f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5\" (UID: \"6efa932f-daab-4dfa-98af-eb221866d49f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"
Jul 09 12:19:12.685082 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:19:12.685058    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d0ebd1d_d802_492e_93d4_340a5574b3c9.slice/crio-c155974fcb417c39a28a34434841b49d0b9d164f56db3a21e3ffa2862d14ce5e WatchSource:0}: Error finding container c155974fcb417c39a28a34434841b49d0b9d164f56db3a21e3ffa2862d14ce5e: Status 404 returned error can't find the container with id c155974fcb417c39a28a34434841b49d0b9d164f56db3a21e3ffa2862d14ce5e
Jul 09 12:19:12.784138 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.784102    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x2pd\" (UniqueName: \"kubernetes.io/projected/6efa932f-daab-4dfa-98af-eb221866d49f-kube-api-access-2x2pd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5\" (UID: \"6efa932f-daab-4dfa-98af-eb221866d49f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"
Jul 09 12:19:12.784427 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.784150    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6efa932f-daab-4dfa-98af-eb221866d49f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5\" (UID: \"6efa932f-daab-4dfa-98af-eb221866d49f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"
Jul 09 12:19:12.784427 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.784198    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6efa932f-daab-4dfa-98af-eb221866d49f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5\" (UID: \"6efa932f-daab-4dfa-98af-eb221866d49f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"
Jul 09 12:19:12.784567 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.784538    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6efa932f-daab-4dfa-98af-eb221866d49f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5\" (UID: \"6efa932f-daab-4dfa-98af-eb221866d49f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"
Jul 09 12:19:12.784622 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.784570    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6efa932f-daab-4dfa-98af-eb221866d49f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5\" (UID: \"6efa932f-daab-4dfa-98af-eb221866d49f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"
Jul 09 12:19:12.792871 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.792807    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x2pd\" (UniqueName: \"kubernetes.io/projected/6efa932f-daab-4dfa-98af-eb221866d49f-kube-api-access-2x2pd\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5\" (UID: \"6efa932f-daab-4dfa-98af-eb221866d49f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"
Jul 09 12:19:12.799293 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.799259    2565 generic.go:356] "Generic (PLEG): container finished" podID="9d0ebd1d-d802-492e-93d4-340a5574b3c9" containerID="7404cbf6c5ff83ccca320677008ad7a8393c0aa97991e03798c008545f65f501" exitCode=0
Jul 09 12:19:12.799462 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.799346    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw" event={"ID":"9d0ebd1d-d802-492e-93d4-340a5574b3c9","Type":"ContainerDied","Data":"7404cbf6c5ff83ccca320677008ad7a8393c0aa97991e03798c008545f65f501"}
Jul 09 12:19:12.799462 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.799396    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw" event={"ID":"9d0ebd1d-d802-492e-93d4-340a5574b3c9","Type":"ContainerStarted","Data":"c155974fcb417c39a28a34434841b49d0b9d164f56db3a21e3ffa2862d14ce5e"}
Jul 09 12:19:12.799818 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.799802    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 12:19:12.945559 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:12.945522    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"
Jul 09 12:19:13.083413 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.083203    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"]
Jul 09 12:19:13.085539 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:19:13.085511    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6efa932f_daab_4dfa_98af_eb221866d49f.slice/crio-c6c0c762d27d9f064a05cc249b2d7acfca82e553c4c9784ad61cd41a87a19643 WatchSource:0}: Error finding container c6c0c762d27d9f064a05cc249b2d7acfca82e553c4c9784ad61cd41a87a19643: Status 404 returned error can't find the container with id c6c0c762d27d9f064a05cc249b2d7acfca82e553c4c9784ad61cd41a87a19643
Jul 09 12:19:13.222292 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.222255    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"]
Jul 09 12:19:13.225726 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.225694    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"
Jul 09 12:19:13.236080 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.236050    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"]
Jul 09 12:19:13.288342 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.288307    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svjcp\" (UniqueName: \"kubernetes.io/projected/7cd144f5-bf25-4752-8353-3c549037a44a-kube-api-access-svjcp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl\" (UID: \"7cd144f5-bf25-4752-8353-3c549037a44a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"
Jul 09 12:19:13.288526 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.288363    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7cd144f5-bf25-4752-8353-3c549037a44a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl\" (UID: \"7cd144f5-bf25-4752-8353-3c549037a44a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"
Jul 09 12:19:13.288526 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.288395    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7cd144f5-bf25-4752-8353-3c549037a44a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl\" (UID: \"7cd144f5-bf25-4752-8353-3c549037a44a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"
Jul 09 12:19:13.389775 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.389662    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svjcp\" (UniqueName: \"kubernetes.io/projected/7cd144f5-bf25-4752-8353-3c549037a44a-kube-api-access-svjcp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl\" (UID: \"7cd144f5-bf25-4752-8353-3c549037a44a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"
Jul 09 12:19:13.389943 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.389781    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7cd144f5-bf25-4752-8353-3c549037a44a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl\" (UID: \"7cd144f5-bf25-4752-8353-3c549037a44a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"
Jul 09 12:19:13.389943 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.389846    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7cd144f5-bf25-4752-8353-3c549037a44a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl\" (UID: \"7cd144f5-bf25-4752-8353-3c549037a44a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"
Jul 09 12:19:13.390222 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.390200    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7cd144f5-bf25-4752-8353-3c549037a44a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl\" (UID: \"7cd144f5-bf25-4752-8353-3c549037a44a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"
Jul 09 12:19:13.390266 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.390210    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7cd144f5-bf25-4752-8353-3c549037a44a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl\" (UID: \"7cd144f5-bf25-4752-8353-3c549037a44a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"
Jul 09 12:19:13.398004 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.397954    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svjcp\" (UniqueName: \"kubernetes.io/projected/7cd144f5-bf25-4752-8353-3c549037a44a-kube-api-access-svjcp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl\" (UID: \"7cd144f5-bf25-4752-8353-3c549037a44a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"
Jul 09 12:19:13.535444 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.535411    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"
Jul 09 12:19:13.755190 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.755171    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"]
Jul 09 12:19:13.757325 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:19:13.757300    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cd144f5_bf25_4752_8353_3c549037a44a.slice/crio-7d8caa069ef720737ca6ab18915566ea197c7e7015d21c83517e0f551aa9bfe8 WatchSource:0}: Error finding container 7d8caa069ef720737ca6ab18915566ea197c7e7015d21c83517e0f551aa9bfe8: Status 404 returned error can't find the container with id 7d8caa069ef720737ca6ab18915566ea197c7e7015d21c83517e0f551aa9bfe8
Jul 09 12:19:13.806245 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.806205    2565 generic.go:356] "Generic (PLEG): container finished" podID="9d0ebd1d-d802-492e-93d4-340a5574b3c9" containerID="bc4e24f8288e5b5450e8dd0196840071a28f6ac1754eff068c01c5f09debf1bf" exitCode=0
Jul 09 12:19:13.806423 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.806289    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw" event={"ID":"9d0ebd1d-d802-492e-93d4-340a5574b3c9","Type":"ContainerDied","Data":"bc4e24f8288e5b5450e8dd0196840071a28f6ac1754eff068c01c5f09debf1bf"}
Jul 09 12:19:13.807939 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.807915    2565 generic.go:356] "Generic (PLEG): container finished" podID="6efa932f-daab-4dfa-98af-eb221866d49f" containerID="1e890b035099b9c1bed4196ab1e8c049b29b52da29f70b1a7cb7193c6815b24d" exitCode=0
Jul 09 12:19:13.808036 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.808008    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5" event={"ID":"6efa932f-daab-4dfa-98af-eb221866d49f","Type":"ContainerDied","Data":"1e890b035099b9c1bed4196ab1e8c049b29b52da29f70b1a7cb7193c6815b24d"}
Jul 09 12:19:13.808100 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.808064    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5" event={"ID":"6efa932f-daab-4dfa-98af-eb221866d49f","Type":"ContainerStarted","Data":"c6c0c762d27d9f064a05cc249b2d7acfca82e553c4c9784ad61cd41a87a19643"}
Jul 09 12:19:13.809331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.809313    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl" event={"ID":"7cd144f5-bf25-4752-8353-3c549037a44a","Type":"ContainerStarted","Data":"7d8caa069ef720737ca6ab18915566ea197c7e7015d21c83517e0f551aa9bfe8"}
Jul 09 12:19:13.822536 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.822510    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"]
Jul 09 12:19:13.829165 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.829134    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"
Jul 09 12:19:13.835344 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.835315    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"]
Jul 09 12:19:13.893869 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.893830    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ch4m\" (UniqueName: \"kubernetes.io/projected/949368a2-d606-447f-83c9-e1a9cfb5798e-kube-api-access-8ch4m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6\" (UID: \"949368a2-d606-447f-83c9-e1a9cfb5798e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"
Jul 09 12:19:13.894036 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.893965    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/949368a2-d606-447f-83c9-e1a9cfb5798e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6\" (UID: \"949368a2-d606-447f-83c9-e1a9cfb5798e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"
Jul 09 12:19:13.894124 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.894076    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/949368a2-d606-447f-83c9-e1a9cfb5798e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6\" (UID: \"949368a2-d606-447f-83c9-e1a9cfb5798e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"
Jul 09 12:19:13.995479 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.995432    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ch4m\" (UniqueName: \"kubernetes.io/projected/949368a2-d606-447f-83c9-e1a9cfb5798e-kube-api-access-8ch4m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6\" (UID: \"949368a2-d606-447f-83c9-e1a9cfb5798e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"
Jul 09 12:19:13.995662 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.995504    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/949368a2-d606-447f-83c9-e1a9cfb5798e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6\" (UID: \"949368a2-d606-447f-83c9-e1a9cfb5798e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"
Jul 09 12:19:13.995662 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.995540    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/949368a2-d606-447f-83c9-e1a9cfb5798e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6\" (UID: \"949368a2-d606-447f-83c9-e1a9cfb5798e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"
Jul 09 12:19:13.995958 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.995934    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/949368a2-d606-447f-83c9-e1a9cfb5798e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6\" (UID: \"949368a2-d606-447f-83c9-e1a9cfb5798e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"
Jul 09 12:19:13.995958 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:13.995950    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/949368a2-d606-447f-83c9-e1a9cfb5798e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6\" (UID: \"949368a2-d606-447f-83c9-e1a9cfb5798e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"
Jul 09 12:19:14.003714 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:14.003673    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ch4m\" (UniqueName: \"kubernetes.io/projected/949368a2-d606-447f-83c9-e1a9cfb5798e-kube-api-access-8ch4m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6\" (UID: \"949368a2-d606-447f-83c9-e1a9cfb5798e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"
Jul 09 12:19:14.163552 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:14.163461    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"
Jul 09 12:19:14.304513 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:14.304484    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"]
Jul 09 12:19:14.306429 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:19:14.306398    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod949368a2_d606_447f_83c9_e1a9cfb5798e.slice/crio-a75d0f4d4d839307dd4219e35ab252e10ee920d4086a6bde5c56405be714abeb WatchSource:0}: Error finding container a75d0f4d4d839307dd4219e35ab252e10ee920d4086a6bde5c56405be714abeb: Status 404 returned error can't find the container with id a75d0f4d4d839307dd4219e35ab252e10ee920d4086a6bde5c56405be714abeb
Jul 09 12:19:14.816319 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:14.816287    2565 generic.go:356] "Generic (PLEG): container finished" podID="6efa932f-daab-4dfa-98af-eb221866d49f" containerID="0643dcc0c6c33a2fbda576e8a9c9192dc9ca5a55c7db6eeaed256d77b1398be1" exitCode=0
Jul 09 12:19:14.816530 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:14.816374    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5" event={"ID":"6efa932f-daab-4dfa-98af-eb221866d49f","Type":"ContainerDied","Data":"0643dcc0c6c33a2fbda576e8a9c9192dc9ca5a55c7db6eeaed256d77b1398be1"}
Jul 09 12:19:14.818071 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:14.818048    2565 generic.go:356] "Generic (PLEG): container finished" podID="949368a2-d606-447f-83c9-e1a9cfb5798e" containerID="dc0de5a1c6aab48beee9e10d2f1fd156d511cc6f5ad07005e6f2cc3782083479" exitCode=0
Jul 09 12:19:14.818139 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:14.818121    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6" event={"ID":"949368a2-d606-447f-83c9-e1a9cfb5798e","Type":"ContainerDied","Data":"dc0de5a1c6aab48beee9e10d2f1fd156d511cc6f5ad07005e6f2cc3782083479"}
Jul 09 12:19:14.818194 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:14.818159    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6" event={"ID":"949368a2-d606-447f-83c9-e1a9cfb5798e","Type":"ContainerStarted","Data":"a75d0f4d4d839307dd4219e35ab252e10ee920d4086a6bde5c56405be714abeb"}
Jul 09 12:19:14.819799 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:14.819773    2565 generic.go:356] "Generic (PLEG): container finished" podID="7cd144f5-bf25-4752-8353-3c549037a44a" containerID="bcfb8058d0af5f7bb24ef33e4ed9403cc908f3a34e746c751e7c65ffa72dc911" exitCode=0
Jul 09 12:19:14.819897 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:14.819858    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl" event={"ID":"7cd144f5-bf25-4752-8353-3c549037a44a","Type":"ContainerDied","Data":"bcfb8058d0af5f7bb24ef33e4ed9403cc908f3a34e746c751e7c65ffa72dc911"}
Jul 09 12:19:14.822461 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:14.822440    2565 generic.go:356] "Generic (PLEG): container finished" podID="9d0ebd1d-d802-492e-93d4-340a5574b3c9" containerID="c5c50974068d0345f0b63bd742a35eb83a410e87a20bf940042f19e6529e7f50" exitCode=0
Jul 09 12:19:14.822551 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:14.822471    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw" event={"ID":"9d0ebd1d-d802-492e-93d4-340a5574b3c9","Type":"ContainerDied","Data":"c5c50974068d0345f0b63bd742a35eb83a410e87a20bf940042f19e6529e7f50"}
Jul 09 12:19:15.830557 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:15.830521    2565 generic.go:356] "Generic (PLEG): container finished" podID="949368a2-d606-447f-83c9-e1a9cfb5798e" containerID="e67679f53a7333cd1ff6efbc343481b45b3855f4799a461b8a2bbee90fc0f535" exitCode=0
Jul 09 12:19:15.830992 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:15.830615    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6" event={"ID":"949368a2-d606-447f-83c9-e1a9cfb5798e","Type":"ContainerDied","Data":"e67679f53a7333cd1ff6efbc343481b45b3855f4799a461b8a2bbee90fc0f535"}
Jul 09 12:19:15.832962 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:15.832930    2565 generic.go:356] "Generic (PLEG): container finished" podID="6efa932f-daab-4dfa-98af-eb221866d49f" containerID="640836a1004700385bf5e95e0d955b2b13e46631b9f6875065701b7a9e8a55a6" exitCode=0
Jul 09 12:19:15.833099 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:15.833002    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5" event={"ID":"6efa932f-daab-4dfa-98af-eb221866d49f","Type":"ContainerDied","Data":"640836a1004700385bf5e95e0d955b2b13e46631b9f6875065701b7a9e8a55a6"}
Jul 09 12:19:15.987878 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:15.987842    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"
Jul 09 12:19:16.115687 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.115650    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d0ebd1d-d802-492e-93d4-340a5574b3c9-bundle\") pod \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\" (UID: \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\") "
Jul 09 12:19:16.115875 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.115815    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsrxc\" (UniqueName: \"kubernetes.io/projected/9d0ebd1d-d802-492e-93d4-340a5574b3c9-kube-api-access-fsrxc\") pod \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\" (UID: \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\") "
Jul 09 12:19:16.115875 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.115837    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d0ebd1d-d802-492e-93d4-340a5574b3c9-util\") pod \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\" (UID: \"9d0ebd1d-d802-492e-93d4-340a5574b3c9\") "
Jul 09 12:19:16.116293 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.116260    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0ebd1d-d802-492e-93d4-340a5574b3c9-bundle" (OuterVolumeSpecName: "bundle") pod "9d0ebd1d-d802-492e-93d4-340a5574b3c9" (UID: "9d0ebd1d-d802-492e-93d4-340a5574b3c9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:19:16.118352 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.118318    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d0ebd1d-d802-492e-93d4-340a5574b3c9-kube-api-access-fsrxc" (OuterVolumeSpecName: "kube-api-access-fsrxc") pod "9d0ebd1d-d802-492e-93d4-340a5574b3c9" (UID: "9d0ebd1d-d802-492e-93d4-340a5574b3c9"). InnerVolumeSpecName "kube-api-access-fsrxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:19:16.121030 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.121005    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0ebd1d-d802-492e-93d4-340a5574b3c9-util" (OuterVolumeSpecName: "util") pod "9d0ebd1d-d802-492e-93d4-340a5574b3c9" (UID: "9d0ebd1d-d802-492e-93d4-340a5574b3c9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:19:16.216982 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.216892    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d0ebd1d-d802-492e-93d4-340a5574b3c9-bundle\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:19:16.216982 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.216924    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsrxc\" (UniqueName: \"kubernetes.io/projected/9d0ebd1d-d802-492e-93d4-340a5574b3c9-kube-api-access-fsrxc\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:19:16.216982 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.216937    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d0ebd1d-d802-492e-93d4-340a5574b3c9-util\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:19:16.839925 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.839897    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw"
Jul 09 12:19:16.840333 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.839922    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw" event={"ID":"9d0ebd1d-d802-492e-93d4-340a5574b3c9","Type":"ContainerDied","Data":"c155974fcb417c39a28a34434841b49d0b9d164f56db3a21e3ffa2862d14ce5e"}
Jul 09 12:19:16.840333 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.840004    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c155974fcb417c39a28a34434841b49d0b9d164f56db3a21e3ffa2862d14ce5e"
Jul 09 12:19:16.842155 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.842130    2565 generic.go:356] "Generic (PLEG): container finished" podID="949368a2-d606-447f-83c9-e1a9cfb5798e" containerID="f78affe28dc1b78675ad5b52acebd9085827d39238c7521da58098ab3ff2e181" exitCode=0
Jul 09 12:19:16.842283 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.842165    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6" event={"ID":"949368a2-d606-447f-83c9-e1a9cfb5798e","Type":"ContainerDied","Data":"f78affe28dc1b78675ad5b52acebd9085827d39238c7521da58098ab3ff2e181"}
Jul 09 12:19:16.843965 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.843940    2565 generic.go:356] "Generic (PLEG): container finished" podID="7cd144f5-bf25-4752-8353-3c549037a44a" containerID="71811ccd0bd70113b1403ca1678091ee804eb40a20871c19342305578e9ad153" exitCode=0
Jul 09 12:19:16.844060 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:16.843972    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl" event={"ID":"7cd144f5-bf25-4752-8353-3c549037a44a","Type":"ContainerDied","Data":"71811ccd0bd70113b1403ca1678091ee804eb40a20871c19342305578e9ad153"}
Jul 09 12:19:17.051130 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.051104    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"
Jul 09 12:19:17.124426 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.124332    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6efa932f-daab-4dfa-98af-eb221866d49f-util\") pod \"6efa932f-daab-4dfa-98af-eb221866d49f\" (UID: \"6efa932f-daab-4dfa-98af-eb221866d49f\") "
Jul 09 12:19:17.124426 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.124408    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x2pd\" (UniqueName: \"kubernetes.io/projected/6efa932f-daab-4dfa-98af-eb221866d49f-kube-api-access-2x2pd\") pod \"6efa932f-daab-4dfa-98af-eb221866d49f\" (UID: \"6efa932f-daab-4dfa-98af-eb221866d49f\") "
Jul 09 12:19:17.124613 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.124465    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6efa932f-daab-4dfa-98af-eb221866d49f-bundle\") pod \"6efa932f-daab-4dfa-98af-eb221866d49f\" (UID: \"6efa932f-daab-4dfa-98af-eb221866d49f\") "
Jul 09 12:19:17.124982 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.124956    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6efa932f-daab-4dfa-98af-eb221866d49f-bundle" (OuterVolumeSpecName: "bundle") pod "6efa932f-daab-4dfa-98af-eb221866d49f" (UID: "6efa932f-daab-4dfa-98af-eb221866d49f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:19:17.126685 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.126661    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6efa932f-daab-4dfa-98af-eb221866d49f-kube-api-access-2x2pd" (OuterVolumeSpecName: "kube-api-access-2x2pd") pod "6efa932f-daab-4dfa-98af-eb221866d49f" (UID: "6efa932f-daab-4dfa-98af-eb221866d49f"). InnerVolumeSpecName "kube-api-access-2x2pd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:19:17.129790 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.129746    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6efa932f-daab-4dfa-98af-eb221866d49f-util" (OuterVolumeSpecName: "util") pod "6efa932f-daab-4dfa-98af-eb221866d49f" (UID: "6efa932f-daab-4dfa-98af-eb221866d49f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:19:17.225125 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.225087    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x2pd\" (UniqueName: \"kubernetes.io/projected/6efa932f-daab-4dfa-98af-eb221866d49f-kube-api-access-2x2pd\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:19:17.225125 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.225119    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6efa932f-daab-4dfa-98af-eb221866d49f-bundle\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:19:17.225125 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.225131    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6efa932f-daab-4dfa-98af-eb221866d49f-util\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:19:17.851422 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.851388    2565 generic.go:356] "Generic (PLEG): container finished" podID="7cd144f5-bf25-4752-8353-3c549037a44a" containerID="adfba38f4c50fdc68d495f92a2292c628a019e293bb0fe547fb39cb2ffffe2ff" exitCode=0
Jul 09 12:19:17.851858 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.851481    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl" event={"ID":"7cd144f5-bf25-4752-8353-3c549037a44a","Type":"ContainerDied","Data":"adfba38f4c50fdc68d495f92a2292c628a019e293bb0fe547fb39cb2ffffe2ff"}
Jul 09 12:19:17.853492 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.853472    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5"
Jul 09 12:19:17.853492 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.853489    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5" event={"ID":"6efa932f-daab-4dfa-98af-eb221866d49f","Type":"ContainerDied","Data":"c6c0c762d27d9f064a05cc249b2d7acfca82e553c4c9784ad61cd41a87a19643"}
Jul 09 12:19:17.853669 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.853528    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6c0c762d27d9f064a05cc249b2d7acfca82e553c4c9784ad61cd41a87a19643"
Jul 09 12:19:17.989409 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:17.989384    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"
Jul 09 12:19:18.132779 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:18.132675    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/949368a2-d606-447f-83c9-e1a9cfb5798e-util\") pod \"949368a2-d606-447f-83c9-e1a9cfb5798e\" (UID: \"949368a2-d606-447f-83c9-e1a9cfb5798e\") "
Jul 09 12:19:18.132779 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:18.132751    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/949368a2-d606-447f-83c9-e1a9cfb5798e-bundle\") pod \"949368a2-d606-447f-83c9-e1a9cfb5798e\" (UID: \"949368a2-d606-447f-83c9-e1a9cfb5798e\") "
Jul 09 12:19:18.132959 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:18.132827    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ch4m\" (UniqueName: \"kubernetes.io/projected/949368a2-d606-447f-83c9-e1a9cfb5798e-kube-api-access-8ch4m\") pod \"949368a2-d606-447f-83c9-e1a9cfb5798e\" (UID: \"949368a2-d606-447f-83c9-e1a9cfb5798e\") "
Jul 09 12:19:18.133445 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:18.133405    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949368a2-d606-447f-83c9-e1a9cfb5798e-bundle" (OuterVolumeSpecName: "bundle") pod "949368a2-d606-447f-83c9-e1a9cfb5798e" (UID: "949368a2-d606-447f-83c9-e1a9cfb5798e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:19:18.135058 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:18.135029    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/949368a2-d606-447f-83c9-e1a9cfb5798e-kube-api-access-8ch4m" (OuterVolumeSpecName: "kube-api-access-8ch4m") pod "949368a2-d606-447f-83c9-e1a9cfb5798e" (UID: "949368a2-d606-447f-83c9-e1a9cfb5798e"). InnerVolumeSpecName "kube-api-access-8ch4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:19:18.138267 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:18.138234    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949368a2-d606-447f-83c9-e1a9cfb5798e-util" (OuterVolumeSpecName: "util") pod "949368a2-d606-447f-83c9-e1a9cfb5798e" (UID: "949368a2-d606-447f-83c9-e1a9cfb5798e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:19:18.234301 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:18.234259    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ch4m\" (UniqueName: \"kubernetes.io/projected/949368a2-d606-447f-83c9-e1a9cfb5798e-kube-api-access-8ch4m\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:19:18.234301 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:18.234295    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/949368a2-d606-447f-83c9-e1a9cfb5798e-util\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:19:18.234301 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:18.234305    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/949368a2-d606-447f-83c9-e1a9cfb5798e-bundle\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:19:18.860616 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:18.860585    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6"
Jul 09 12:19:18.860616 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:18.860605    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6" event={"ID":"949368a2-d606-447f-83c9-e1a9cfb5798e","Type":"ContainerDied","Data":"a75d0f4d4d839307dd4219e35ab252e10ee920d4086a6bde5c56405be714abeb"}
Jul 09 12:19:18.861171 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:18.860649    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a75d0f4d4d839307dd4219e35ab252e10ee920d4086a6bde5c56405be714abeb"
Jul 09 12:19:18.999460 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:18.999434    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"
Jul 09 12:19:19.142332 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:19.142242    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7cd144f5-bf25-4752-8353-3c549037a44a-bundle\") pod \"7cd144f5-bf25-4752-8353-3c549037a44a\" (UID: \"7cd144f5-bf25-4752-8353-3c549037a44a\") "
Jul 09 12:19:19.142332 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:19.142333    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7cd144f5-bf25-4752-8353-3c549037a44a-util\") pod \"7cd144f5-bf25-4752-8353-3c549037a44a\" (UID: \"7cd144f5-bf25-4752-8353-3c549037a44a\") "
Jul 09 12:19:19.142556 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:19.142396    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svjcp\" (UniqueName: \"kubernetes.io/projected/7cd144f5-bf25-4752-8353-3c549037a44a-kube-api-access-svjcp\") pod \"7cd144f5-bf25-4752-8353-3c549037a44a\" (UID: \"7cd144f5-bf25-4752-8353-3c549037a44a\") "
Jul 09 12:19:19.142806 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:19.142778    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd144f5-bf25-4752-8353-3c549037a44a-bundle" (OuterVolumeSpecName: "bundle") pod "7cd144f5-bf25-4752-8353-3c549037a44a" (UID: "7cd144f5-bf25-4752-8353-3c549037a44a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:19:19.144720 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:19.144685    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cd144f5-bf25-4752-8353-3c549037a44a-kube-api-access-svjcp" (OuterVolumeSpecName: "kube-api-access-svjcp") pod "7cd144f5-bf25-4752-8353-3c549037a44a" (UID: "7cd144f5-bf25-4752-8353-3c549037a44a"). InnerVolumeSpecName "kube-api-access-svjcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:19:19.212555 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:19.212504    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cd144f5-bf25-4752-8353-3c549037a44a-util" (OuterVolumeSpecName: "util") pod "7cd144f5-bf25-4752-8353-3c549037a44a" (UID: "7cd144f5-bf25-4752-8353-3c549037a44a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:19:19.243022 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:19.242993    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7cd144f5-bf25-4752-8353-3c549037a44a-util\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:19:19.243022 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:19.243019    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svjcp\" (UniqueName: \"kubernetes.io/projected/7cd144f5-bf25-4752-8353-3c549037a44a-kube-api-access-svjcp\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:19:19.243022 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:19.243029    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7cd144f5-bf25-4752-8353-3c549037a44a-bundle\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:19:19.868162 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:19.868135    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl"
Jul 09 12:19:19.868162 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:19.868155    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl" event={"ID":"7cd144f5-bf25-4752-8353-3c549037a44a","Type":"ContainerDied","Data":"7d8caa069ef720737ca6ab18915566ea197c7e7015d21c83517e0f551aa9bfe8"}
Jul 09 12:19:19.868604 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:19.868222    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d8caa069ef720737ca6ab18915566ea197c7e7015d21c83517e0f551aa9bfe8"
Jul 09 12:19:40.316628 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.316582    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"]
Jul 09 12:19:40.317056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.316987    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cd144f5-bf25-4752-8353-3c549037a44a" containerName="pull"
Jul 09 12:19:40.317056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317003    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cd144f5-bf25-4752-8353-3c549037a44a" containerName="pull"
Jul 09 12:19:40.317056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317018    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d0ebd1d-d802-492e-93d4-340a5574b3c9" containerName="pull"
Jul 09 12:19:40.317056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317024    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d0ebd1d-d802-492e-93d4-340a5574b3c9" containerName="pull"
Jul 09 12:19:40.317056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317029    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="949368a2-d606-447f-83c9-e1a9cfb5798e" containerName="util"
Jul 09 12:19:40.317056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317034    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="949368a2-d606-447f-83c9-e1a9cfb5798e" containerName="util"
Jul 09 12:19:40.317056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317039    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6efa932f-daab-4dfa-98af-eb221866d49f" containerName="pull"
Jul 09 12:19:40.317056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317045    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6efa932f-daab-4dfa-98af-eb221866d49f" containerName="pull"
Jul 09 12:19:40.317056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317051    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="949368a2-d606-447f-83c9-e1a9cfb5798e" containerName="pull"
Jul 09 12:19:40.317056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317056    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="949368a2-d606-447f-83c9-e1a9cfb5798e" containerName="pull"
Jul 09 12:19:40.317056 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317064    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d0ebd1d-d802-492e-93d4-340a5574b3c9" containerName="extract"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317069    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d0ebd1d-d802-492e-93d4-340a5574b3c9" containerName="extract"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317081    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cd144f5-bf25-4752-8353-3c549037a44a" containerName="util"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317088    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cd144f5-bf25-4752-8353-3c549037a44a" containerName="util"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317098    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6efa932f-daab-4dfa-98af-eb221866d49f" containerName="extract"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317105    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6efa932f-daab-4dfa-98af-eb221866d49f" containerName="extract"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317113    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="949368a2-d606-447f-83c9-e1a9cfb5798e" containerName="extract"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317118    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="949368a2-d606-447f-83c9-e1a9cfb5798e" containerName="extract"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317128    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6efa932f-daab-4dfa-98af-eb221866d49f" containerName="util"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317133    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6efa932f-daab-4dfa-98af-eb221866d49f" containerName="util"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317138    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d0ebd1d-d802-492e-93d4-340a5574b3c9" containerName="util"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317143    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d0ebd1d-d802-492e-93d4-340a5574b3c9" containerName="util"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317148    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cd144f5-bf25-4752-8353-3c549037a44a" containerName="extract"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317153    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cd144f5-bf25-4752-8353-3c549037a44a" containerName="extract"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317225    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="949368a2-d606-447f-83c9-e1a9cfb5798e" containerName="extract"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317234    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d0ebd1d-d802-492e-93d4-340a5574b3c9" containerName="extract"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317242    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cd144f5-bf25-4752-8353-3c549037a44a" containerName="extract"
Jul 09 12:19:40.317550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.317252    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6efa932f-daab-4dfa-98af-eb221866d49f" containerName="extract"
Jul 09 12:19:40.320042 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.320023    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"
Jul 09 12:19:40.323052 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.323020    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 09 12:19:40.323201 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.323055    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 12:19:40.323201 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.323148    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 12:19:40.323317 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.323148    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 09 12:19:40.324274 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.324256    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-p6dx7\""
Jul 09 12:19:40.325489 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.325466    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/14f9643e-3daa-478e-824f-2a59d7d86877-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xf9np\" (UID: \"14f9643e-3daa-478e-824f-2a59d7d86877\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"
Jul 09 12:19:40.325586 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.325528    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5djb\" (UniqueName: \"kubernetes.io/projected/14f9643e-3daa-478e-824f-2a59d7d86877-kube-api-access-d5djb\") pod \"kuadrant-console-plugin-66b5774985-xf9np\" (UID: \"14f9643e-3daa-478e-824f-2a59d7d86877\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"
Jul 09 12:19:40.325586 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.325565    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/14f9643e-3daa-478e-824f-2a59d7d86877-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xf9np\" (UID: \"14f9643e-3daa-478e-824f-2a59d7d86877\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"
Jul 09 12:19:40.331006 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.330984    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"]
Jul 09 12:19:40.426692 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.426651    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/14f9643e-3daa-478e-824f-2a59d7d86877-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xf9np\" (UID: \"14f9643e-3daa-478e-824f-2a59d7d86877\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"
Jul 09 12:19:40.426900 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.426734    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5djb\" (UniqueName: \"kubernetes.io/projected/14f9643e-3daa-478e-824f-2a59d7d86877-kube-api-access-d5djb\") pod \"kuadrant-console-plugin-66b5774985-xf9np\" (UID: \"14f9643e-3daa-478e-824f-2a59d7d86877\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"
Jul 09 12:19:40.426900 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.426762    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/14f9643e-3daa-478e-824f-2a59d7d86877-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xf9np\" (UID: \"14f9643e-3daa-478e-824f-2a59d7d86877\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"
Jul 09 12:19:40.426900 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:19:40.426778    2565 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 09 12:19:40.426900 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:19:40.426849    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14f9643e-3daa-478e-824f-2a59d7d86877-plugin-serving-cert podName:14f9643e-3daa-478e-824f-2a59d7d86877 nodeName:}" failed. No retries permitted until 2026-07-09 12:19:40.926833312 +0000 UTC m=+1053.604362348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/14f9643e-3daa-478e-824f-2a59d7d86877-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-xf9np" (UID: "14f9643e-3daa-478e-824f-2a59d7d86877") : secret "plugin-serving-cert" not found
Jul 09 12:19:40.427354 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.427337    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/14f9643e-3daa-478e-824f-2a59d7d86877-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-xf9np\" (UID: \"14f9643e-3daa-478e-824f-2a59d7d86877\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"
Jul 09 12:19:40.435450 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.435427    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5djb\" (UniqueName: \"kubernetes.io/projected/14f9643e-3daa-478e-824f-2a59d7d86877-kube-api-access-d5djb\") pod \"kuadrant-console-plugin-66b5774985-xf9np\" (UID: \"14f9643e-3daa-478e-824f-2a59d7d86877\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"
Jul 09 12:19:40.929587 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.929546    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/14f9643e-3daa-478e-824f-2a59d7d86877-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xf9np\" (UID: \"14f9643e-3daa-478e-824f-2a59d7d86877\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"
Jul 09 12:19:40.932102 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:40.932075    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/14f9643e-3daa-478e-824f-2a59d7d86877-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-xf9np\" (UID: \"14f9643e-3daa-478e-824f-2a59d7d86877\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"
Jul 09 12:19:41.229770 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:41.229670    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"
Jul 09 12:19:41.364381 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:41.364356    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np"]
Jul 09 12:19:41.366105 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:19:41.366067    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14f9643e_3daa_478e_824f_2a59d7d86877.slice/crio-583f5df1802d2a2b1eb54683007a17ac02613c120da74a3469f12f20313c9c22 WatchSource:0}: Error finding container 583f5df1802d2a2b1eb54683007a17ac02613c120da74a3469f12f20313c9c22: Status 404 returned error can't find the container with id 583f5df1802d2a2b1eb54683007a17ac02613c120da74a3469f12f20313c9c22
Jul 09 12:19:41.980385 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:19:41.980335    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np" event={"ID":"14f9643e-3daa-478e-824f-2a59d7d86877","Type":"ContainerStarted","Data":"583f5df1802d2a2b1eb54683007a17ac02613c120da74a3469f12f20313c9c22"}
Jul 09 12:20:05.125125 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:05.125085    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np" event={"ID":"14f9643e-3daa-478e-824f-2a59d7d86877","Type":"ContainerStarted","Data":"361a13b792638b3b0b83488081570050e1c1956e7f72a2ea52c507e57d34ed97"}
Jul 09 12:20:05.146523 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:05.146458    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-xf9np" podStartSLOduration=1.511296662 podStartE2EDuration="25.146440808s" totalImagesPullingTime="23.635144146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:20:05.143753276 +0000 UTC m=+1077.821282336" watchObservedRunningTime="2026-07-09 12:20:05.146440808 +0000 UTC m=+1077.823969857"
Jul 09 12:20:19.989986 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:19.989948    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"]
Jul 09 12:20:20.005952 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.005916    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.008873 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.008844    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-m8j6n\""
Jul 09 12:20:20.009750 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.009688    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"]
Jul 09 12:20:20.071529 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.071487    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.071529 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.071537    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.071841 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.071644    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df01513e-f269-4c17-8272-b80b59e1810a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.071841 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.071751    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.071841 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.071790    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.071841 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.071815    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df01513e-f269-4c17-8272-b80b59e1810a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.072034 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.071856    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df01513e-f269-4c17-8272-b80b59e1810a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.072034 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.071880    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cbsh\" (UniqueName: \"kubernetes.io/projected/df01513e-f269-4c17-8272-b80b59e1810a-kube-api-access-6cbsh\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.072034 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.071910    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.172368 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.172332    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df01513e-f269-4c17-8272-b80b59e1810a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.172568 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.172379    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df01513e-f269-4c17-8272-b80b59e1810a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.172568 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.172398    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cbsh\" (UniqueName: \"kubernetes.io/projected/df01513e-f269-4c17-8272-b80b59e1810a-kube-api-access-6cbsh\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.172568 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.172426    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.172568 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.172457    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.172568 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.172494    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.172568 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.172539    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df01513e-f269-4c17-8272-b80b59e1810a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.172965 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.172677    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.172965 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.172766    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.173079 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.173056    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.173149 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.173116    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.173212 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.173120    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.173212 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.173189    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df01513e-f269-4c17-8272-b80b59e1810a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.173480 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.173197    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.175012 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.174985    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df01513e-f269-4c17-8272-b80b59e1810a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.175281 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.175260    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df01513e-f269-4c17-8272-b80b59e1810a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.182080 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.182047    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df01513e-f269-4c17-8272-b80b59e1810a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.182275 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.182251    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cbsh\" (UniqueName: \"kubernetes.io/projected/df01513e-f269-4c17-8272-b80b59e1810a-kube-api-access-6cbsh\") pod \"maas-default-gateway-openshift-default-56654dcf7-t8nbr\" (UID: \"df01513e-f269-4c17-8272-b80b59e1810a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.317968 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.317941    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:20.462154 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.462124    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"]
Jul 09 12:20:20.463912 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:20:20.463890    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf01513e_f269_4c17_8272_b80b59e1810a.slice/crio-cd3bd4fb08ed0482e044281905f235b658fd5e43e0f5acec884dbf50190e5047 WatchSource:0}: Error finding container cd3bd4fb08ed0482e044281905f235b658fd5e43e0f5acec884dbf50190e5047: Status 404 returned error can't find the container with id cd3bd4fb08ed0482e044281905f235b658fd5e43e0f5acec884dbf50190e5047
Jul 09 12:20:20.465982 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.465950    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:20:20.466067 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.466019    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:20:20.466067 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:20.466048    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:20:21.226736 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:21.226672    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr" event={"ID":"df01513e-f269-4c17-8272-b80b59e1810a","Type":"ContainerStarted","Data":"64ca742ea29ff2f66187fece6511532bf732c6a29b99afea90f7fe3a1931675e"}
Jul 09 12:20:21.227144 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:21.226782    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr" event={"ID":"df01513e-f269-4c17-8272-b80b59e1810a","Type":"ContainerStarted","Data":"cd3bd4fb08ed0482e044281905f235b658fd5e43e0f5acec884dbf50190e5047"}
Jul 09 12:20:21.248783 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:21.248730    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr" podStartSLOduration=2.248692568 podStartE2EDuration="2.248692568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:20:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:20:21.247930354 +0000 UTC m=+1093.925459412" watchObservedRunningTime="2026-07-09 12:20:21.248692568 +0000 UTC m=+1093.926221626"
Jul 09 12:20:21.318511 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:21.318475    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:21.323304 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:21.323278    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:22.233040 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:22.233002    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:22.234095 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:22.234076    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:20:22.750165 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:22.750130    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:20:22.753808 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:22.753791    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"
Jul 09 12:20:22.756872 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:22.756852    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 09 12:20:22.764860 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:22.764833    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:20:22.801959 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:22.801925    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:20:22.897990 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:22.897953    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7bc591db-6b3f-4d74-9567-90f8890c76c5-config-file\") pod \"limitador-limitador-54d6d988d5-v4cbc\" (UID: \"7bc591db-6b3f-4d74-9567-90f8890c76c5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"
Jul 09 12:20:22.898144 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:22.897999    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t25br\" (UniqueName: \"kubernetes.io/projected/7bc591db-6b3f-4d74-9567-90f8890c76c5-kube-api-access-t25br\") pod \"limitador-limitador-54d6d988d5-v4cbc\" (UID: \"7bc591db-6b3f-4d74-9567-90f8890c76c5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"
Jul 09 12:20:22.999456 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:22.999414    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7bc591db-6b3f-4d74-9567-90f8890c76c5-config-file\") pod \"limitador-limitador-54d6d988d5-v4cbc\" (UID: \"7bc591db-6b3f-4d74-9567-90f8890c76c5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"
Jul 09 12:20:22.999456 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:22.999470    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t25br\" (UniqueName: \"kubernetes.io/projected/7bc591db-6b3f-4d74-9567-90f8890c76c5-kube-api-access-t25br\") pod \"limitador-limitador-54d6d988d5-v4cbc\" (UID: \"7bc591db-6b3f-4d74-9567-90f8890c76c5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"
Jul 09 12:20:23.000355 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:23.000303    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7bc591db-6b3f-4d74-9567-90f8890c76c5-config-file\") pod \"limitador-limitador-54d6d988d5-v4cbc\" (UID: \"7bc591db-6b3f-4d74-9567-90f8890c76c5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"
Jul 09 12:20:23.024643 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:23.024608    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t25br\" (UniqueName: \"kubernetes.io/projected/7bc591db-6b3f-4d74-9567-90f8890c76c5-kube-api-access-t25br\") pod \"limitador-limitador-54d6d988d5-v4cbc\" (UID: \"7bc591db-6b3f-4d74-9567-90f8890c76c5\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"
Jul 09 12:20:23.065142 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:23.065109    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"
Jul 09 12:20:23.212690 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:23.212655    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:20:23.214296 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:20:23.214250    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bc591db_6b3f_4d74_9567_90f8890c76c5.slice/crio-cf1a80a76fc90f6a30a06d1966e3c2cbb3c850e51e340d151777200fa06bca3a WatchSource:0}: Error finding container cf1a80a76fc90f6a30a06d1966e3c2cbb3c850e51e340d151777200fa06bca3a: Status 404 returned error can't find the container with id cf1a80a76fc90f6a30a06d1966e3c2cbb3c850e51e340d151777200fa06bca3a
Jul 09 12:20:23.238847 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:23.238818    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4cbc" event={"ID":"7bc591db-6b3f-4d74-9567-90f8890c76c5","Type":"ContainerStarted","Data":"cf1a80a76fc90f6a30a06d1966e3c2cbb3c850e51e340d151777200fa06bca3a"}
Jul 09 12:20:27.266290 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:27.266243    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4cbc" event={"ID":"7bc591db-6b3f-4d74-9567-90f8890c76c5","Type":"ContainerStarted","Data":"3fdd429928f406676a6efdde332c4674f8b99df5a860392663be01e4c6f479b6"}
Jul 09 12:20:27.266743 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:27.266320    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"
Jul 09 12:20:27.291781 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:27.291673    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4cbc" podStartSLOduration=1.48229939 podStartE2EDuration="5.291657221s" totalImagesPullingTime="3.809357831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:20:27.288871906 +0000 UTC m=+1099.966400976" watchObservedRunningTime="2026-07-09 12:20:27.291657221 +0000 UTC m=+1099.969186300"
Jul 09 12:20:38.272973 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:38.272942    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"
Jul 09 12:20:56.744091 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:56.744042    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-755cdc4b99-v47m5"]
Jul 09 12:20:56.760903 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:56.760874    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-755cdc4b99-v47m5"]
Jul 09 12:20:56.761051 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:56.760972    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-755cdc4b99-v47m5"
Jul 09 12:20:56.763858 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:56.763838    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rpqb7\""
Jul 09 12:20:56.792237 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:56.792205    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhm9w\" (UniqueName: \"kubernetes.io/projected/0642e50f-e196-4dbb-b671-5c6ea53c3cfd-kube-api-access-zhm9w\") pod \"authorino-755cdc4b99-v47m5\" (UID: \"0642e50f-e196-4dbb-b671-5c6ea53c3cfd\") " pod="kuadrant-system/authorino-755cdc4b99-v47m5"
Jul 09 12:20:56.844795 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:56.844754    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-755cdc4b99-v47m5"]
Jul 09 12:20:56.845049 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:20:56.845028    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-zhm9w], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-755cdc4b99-v47m5" podUID="0642e50f-e196-4dbb-b671-5c6ea53c3cfd"
Jul 09 12:20:56.892883 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:56.892851    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhm9w\" (UniqueName: \"kubernetes.io/projected/0642e50f-e196-4dbb-b671-5c6ea53c3cfd-kube-api-access-zhm9w\") pod \"authorino-755cdc4b99-v47m5\" (UID: \"0642e50f-e196-4dbb-b671-5c6ea53c3cfd\") " pod="kuadrant-system/authorino-755cdc4b99-v47m5"
Jul 09 12:20:56.900939 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:56.900911    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhm9w\" (UniqueName: \"kubernetes.io/projected/0642e50f-e196-4dbb-b671-5c6ea53c3cfd-kube-api-access-zhm9w\") pod \"authorino-755cdc4b99-v47m5\" (UID: \"0642e50f-e196-4dbb-b671-5c6ea53c3cfd\") " pod="kuadrant-system/authorino-755cdc4b99-v47m5"
Jul 09 12:20:57.423509 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:57.423475    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-755cdc4b99-v47m5"
Jul 09 12:20:57.432607 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:57.432581    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-755cdc4b99-v47m5"
Jul 09 12:20:57.498103 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:57.498067    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhm9w\" (UniqueName: \"kubernetes.io/projected/0642e50f-e196-4dbb-b671-5c6ea53c3cfd-kube-api-access-zhm9w\") pod \"0642e50f-e196-4dbb-b671-5c6ea53c3cfd\" (UID: \"0642e50f-e196-4dbb-b671-5c6ea53c3cfd\") "
Jul 09 12:20:57.500423 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:57.500384    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0642e50f-e196-4dbb-b671-5c6ea53c3cfd-kube-api-access-zhm9w" (OuterVolumeSpecName: "kube-api-access-zhm9w") pod "0642e50f-e196-4dbb-b671-5c6ea53c3cfd" (UID: "0642e50f-e196-4dbb-b671-5c6ea53c3cfd"). InnerVolumeSpecName "kube-api-access-zhm9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:20:57.599678 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:57.599641    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhm9w\" (UniqueName: \"kubernetes.io/projected/0642e50f-e196-4dbb-b671-5c6ea53c3cfd-kube-api-access-zhm9w\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:20:58.434643 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:58.434610    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-755cdc4b99-v47m5"
Jul 09 12:20:58.463168 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:58.463135    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-755cdc4b99-v47m5"]
Jul 09 12:20:58.467816 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:58.467786    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-755cdc4b99-v47m5"]
Jul 09 12:20:59.894715 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:20:59.894675    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0642e50f-e196-4dbb-b671-5c6ea53c3cfd" path="/var/lib/kubelet/pods/0642e50f-e196-4dbb-b671-5c6ea53c3cfd/volumes"
Jul 09 12:21:39.220452 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.220408    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"]
Jul 09 12:21:39.223304 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.223278    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"
Jul 09 12:21:39.228634 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.228606    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 12:21:39.230032 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.229900    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-52h8m\""
Jul 09 12:21:39.230161 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.230138    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 12:21:39.230308 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.230204    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 09 12:21:39.242151 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.242125    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"]
Jul 09 12:21:39.279557 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.279526    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5594e249-da08-4e28-88ef-80096591b5fe-maas-api-tls\") pod \"maas-api-7fd5f457bd-ppsrm\" (UID: \"5594e249-da08-4e28-88ef-80096591b5fe\") " pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"
Jul 09 12:21:39.279779 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.279757    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v76zg\" (UniqueName: \"kubernetes.io/projected/5594e249-da08-4e28-88ef-80096591b5fe-kube-api-access-v76zg\") pod \"maas-api-7fd5f457bd-ppsrm\" (UID: \"5594e249-da08-4e28-88ef-80096591b5fe\") " pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"
Jul 09 12:21:39.279881 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.279861    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5594e249-da08-4e28-88ef-80096591b5fe-openshift-service-ca\") pod \"maas-api-7fd5f457bd-ppsrm\" (UID: \"5594e249-da08-4e28-88ef-80096591b5fe\") " pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"
Jul 09 12:21:39.380689 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.380641    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5594e249-da08-4e28-88ef-80096591b5fe-openshift-service-ca\") pod \"maas-api-7fd5f457bd-ppsrm\" (UID: \"5594e249-da08-4e28-88ef-80096591b5fe\") " pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"
Jul 09 12:21:39.380893 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.380764    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5594e249-da08-4e28-88ef-80096591b5fe-maas-api-tls\") pod \"maas-api-7fd5f457bd-ppsrm\" (UID: \"5594e249-da08-4e28-88ef-80096591b5fe\") " pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"
Jul 09 12:21:39.380893 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.380795    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v76zg\" (UniqueName: \"kubernetes.io/projected/5594e249-da08-4e28-88ef-80096591b5fe-kube-api-access-v76zg\") pod \"maas-api-7fd5f457bd-ppsrm\" (UID: \"5594e249-da08-4e28-88ef-80096591b5fe\") " pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"
Jul 09 12:21:39.381340 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.381304    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5594e249-da08-4e28-88ef-80096591b5fe-openshift-service-ca\") pod \"maas-api-7fd5f457bd-ppsrm\" (UID: \"5594e249-da08-4e28-88ef-80096591b5fe\") " pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"
Jul 09 12:21:39.383384 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.383363    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5594e249-da08-4e28-88ef-80096591b5fe-maas-api-tls\") pod \"maas-api-7fd5f457bd-ppsrm\" (UID: \"5594e249-da08-4e28-88ef-80096591b5fe\") " pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"
Jul 09 12:21:39.391005 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.390981    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v76zg\" (UniqueName: \"kubernetes.io/projected/5594e249-da08-4e28-88ef-80096591b5fe-kube-api-access-v76zg\") pod \"maas-api-7fd5f457bd-ppsrm\" (UID: \"5594e249-da08-4e28-88ef-80096591b5fe\") " pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"
Jul 09 12:21:39.533973 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.533876    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"
Jul 09 12:21:39.679809 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.679783    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"]
Jul 09 12:21:39.682456 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:21:39.682424    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5594e249_da08_4e28_88ef_80096591b5fe.slice/crio-988e8769a6c0b02e20632f0ad7c5952cc21f37a1fd2c53a6d6795ed03eb623e1 WatchSource:0}: Error finding container 988e8769a6c0b02e20632f0ad7c5952cc21f37a1fd2c53a6d6795ed03eb623e1: Status 404 returned error can't find the container with id 988e8769a6c0b02e20632f0ad7c5952cc21f37a1fd2c53a6d6795ed03eb623e1
Jul 09 12:21:39.836714 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:39.836657    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:21:40.656149 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:40.656107    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm" event={"ID":"5594e249-da08-4e28-88ef-80096591b5fe","Type":"ContainerStarted","Data":"988e8769a6c0b02e20632f0ad7c5952cc21f37a1fd2c53a6d6795ed03eb623e1"}
Jul 09 12:21:42.668195 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:42.668155    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm" event={"ID":"5594e249-da08-4e28-88ef-80096591b5fe","Type":"ContainerStarted","Data":"b32f855e58be133abce1bd359e3d12ed4902edf38be59897e9a03cfb651dc022"}
Jul 09 12:21:42.668584 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:42.668217    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"
Jul 09 12:21:42.686766 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:42.686693    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm" podStartSLOduration=1.52250072 podStartE2EDuration="3.686676909s" totalImagesPullingTime="2.164176189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:21:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:21:42.684946452 +0000 UTC m=+1175.362475613" watchObservedRunningTime="2026-07-09 12:21:42.686676909 +0000 UTC m=+1175.364205966"
Jul 09 12:21:47.697223 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:47.697190    2565 generic.go:356] "Generic (PLEG): container finished" podID="df01513e-f269-4c17-8272-b80b59e1810a" containerID="64ca742ea29ff2f66187fece6511532bf732c6a29b99afea90f7fe3a1931675e" exitCode=0
Jul 09 12:21:47.697637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:47.697266    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr" event={"ID":"df01513e-f269-4c17-8272-b80b59e1810a","Type":"ContainerDied","Data":"64ca742ea29ff2f66187fece6511532bf732c6a29b99afea90f7fe3a1931675e"}
Jul 09 12:21:47.697720 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:47.697683    2565 scope.go:117] "RemoveContainer" containerID="64ca742ea29ff2f66187fece6511532bf732c6a29b99afea90f7fe3a1931675e"
Jul 09 12:21:47.698745 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:47.698717    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:21:47.698819 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:47.698799    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:21:47.698856 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:47.698845    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:21:48.680219 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:48.680184    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7fd5f457bd-ppsrm"
Jul 09 12:21:48.706230 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:48.706196    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr" event={"ID":"df01513e-f269-4c17-8272-b80b59e1810a","Type":"ContainerStarted","Data":"75d19db4ca89f0752b96bb77940de2d5ab25e9a0f18128014ae1f911d578673f"}
Jul 09 12:21:49.318833 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:49.318793    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:21:49.320029 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:49.319995    2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-t8nbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 09 12:21:49.320238 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:49.320212    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr" podUID="df01513e-f269-4c17-8272-b80b59e1810a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 09 12:21:50.318729 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:50.318673    2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-t8nbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 09 12:21:50.319242 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:50.318754    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr" podUID="df01513e-f269-4c17-8272-b80b59e1810a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 09 12:21:51.319230 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:51.319178    2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-t8nbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 09 12:21:51.319767 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:51.319253    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr" podUID="df01513e-f269-4c17-8272-b80b59e1810a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 09 12:21:52.233391 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:52.233345    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:21:52.319033 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:52.318992    2565 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-t8nbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 09 12:21:52.319242 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:52.319063    2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr" podUID="df01513e-f269-4c17-8272-b80b59e1810a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 09 12:21:53.323031 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:53.322995    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:21:53.324120 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:21:53.324097    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t8nbr"
Jul 09 12:22:03.919156 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:03.919117    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:22:07.869427 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:07.869391    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:22:07.869872 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:07.869634    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:22:09.519442 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:09.519404    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:22:31.925589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:31.925556    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"]
Jul 09 12:22:31.929222 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:31.929205    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:31.932112 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:31.932090    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 09 12:22:31.933265 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:31.933245    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 09 12:22:31.933496 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:31.933481    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sgjkl\""
Jul 09 12:22:31.933583 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:31.933521    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 09 12:22:31.938461 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:31.938438    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"]
Jul 09 12:22:31.974782 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:31.974744    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fff3852-dc80-40bf-9732-90f525bc0729-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:31.974782 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:31.974783    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7fzm\" (UniqueName: \"kubernetes.io/projected/6fff3852-dc80-40bf-9732-90f525bc0729-kube-api-access-c7fzm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:31.975043 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:31.974804    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fff3852-dc80-40bf-9732-90f525bc0729-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:31.975043 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:31.974855    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fff3852-dc80-40bf-9732-90f525bc0729-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:31.975043 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:31.974900    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fff3852-dc80-40bf-9732-90f525bc0729-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:31.975043 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:31.975029    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fff3852-dc80-40bf-9732-90f525bc0729-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:32.075569 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.075533    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fff3852-dc80-40bf-9732-90f525bc0729-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:32.075809 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.075631    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fff3852-dc80-40bf-9732-90f525bc0729-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:32.075809 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.075660    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7fzm\" (UniqueName: \"kubernetes.io/projected/6fff3852-dc80-40bf-9732-90f525bc0729-kube-api-access-c7fzm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:32.075809 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.075689    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fff3852-dc80-40bf-9732-90f525bc0729-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:32.075809 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.075744    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fff3852-dc80-40bf-9732-90f525bc0729-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:32.075809 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.075767    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fff3852-dc80-40bf-9732-90f525bc0729-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:32.076086 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.076046    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fff3852-dc80-40bf-9732-90f525bc0729-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:32.076140 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.076087    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fff3852-dc80-40bf-9732-90f525bc0729-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:32.076140 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.076113    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fff3852-dc80-40bf-9732-90f525bc0729-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:32.078086 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.078069    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fff3852-dc80-40bf-9732-90f525bc0729-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:32.078216 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.078198    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fff3852-dc80-40bf-9732-90f525bc0729-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:32.083954 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.083928    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7fzm\" (UniqueName: \"kubernetes.io/projected/6fff3852-dc80-40bf-9732-90f525bc0729-kube-api-access-c7fzm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vhgl4\" (UID: \"6fff3852-dc80-40bf-9732-90f525bc0729\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:32.241798 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.241655    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:32.382779 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.382747    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"]
Jul 09 12:22:32.384518 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:22:32.384492    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fff3852_dc80_40bf_9732_90f525bc0729.slice/crio-bcc569dc6b9207032f1f6f49f2f7965118c06a03c469b99e879a9c6d244da2b7 WatchSource:0}: Error finding container bcc569dc6b9207032f1f6f49f2f7965118c06a03c469b99e879a9c6d244da2b7: Status 404 returned error can't find the container with id bcc569dc6b9207032f1f6f49f2f7965118c06a03c469b99e879a9c6d244da2b7
Jul 09 12:22:32.949098 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:32.949057    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4" event={"ID":"6fff3852-dc80-40bf-9732-90f525bc0729","Type":"ContainerStarted","Data":"bcc569dc6b9207032f1f6f49f2f7965118c06a03c469b99e879a9c6d244da2b7"}
Jul 09 12:22:33.130170 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:33.130133    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:22:33.955785 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:33.955744    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4" event={"ID":"6fff3852-dc80-40bf-9732-90f525bc0729","Type":"ContainerStarted","Data":"b6828471295b33440e9a1113a5311718ff0aff387ed77103ca1d0d250c9519c8"}
Jul 09 12:22:33.975995 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:33.975934    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4" podStartSLOduration=1.814712295 podStartE2EDuration="2.975917088s" totalImagesPullingTime="1.161204793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:22:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:22:33.974437275 +0000 UTC m=+1226.651966338" watchObservedRunningTime="2026-07-09 12:22:33.975917088 +0000 UTC m=+1226.653446124"
Jul 09 12:22:42.242062 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:42.242004    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:42.242600 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:42.242093    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:42.255343 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:42.255318    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:43.014825 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:43.014796    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vhgl4"
Jul 09 12:22:47.527897 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:47.527855    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:22:51.508541 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.508509    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"]
Jul 09 12:22:51.512256 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.512238    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.518198 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.518173    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 09 12:22:51.530273 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.530238    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"]
Jul 09 12:22:51.658899 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.658863    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4685e84e-edb6-4ea1-b396-db534f083bcd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.659119 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.658920    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m552s\" (UniqueName: \"kubernetes.io/projected/4685e84e-edb6-4ea1-b396-db534f083bcd-kube-api-access-m552s\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.659119 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.658960    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4685e84e-edb6-4ea1-b396-db534f083bcd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.659119 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.659055    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4685e84e-edb6-4ea1-b396-db534f083bcd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.659119 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.659081    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4685e84e-edb6-4ea1-b396-db534f083bcd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.659119 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.659108    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4685e84e-edb6-4ea1-b396-db534f083bcd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.760444 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.760333    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4685e84e-edb6-4ea1-b396-db534f083bcd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.760444 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.760395    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4685e84e-edb6-4ea1-b396-db534f083bcd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.760444 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.760439    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m552s\" (UniqueName: \"kubernetes.io/projected/4685e84e-edb6-4ea1-b396-db534f083bcd-kube-api-access-m552s\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.760801 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.760467    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4685e84e-edb6-4ea1-b396-db534f083bcd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.760801 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.760536    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4685e84e-edb6-4ea1-b396-db534f083bcd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.760801 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.760559    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4685e84e-edb6-4ea1-b396-db534f083bcd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.760975 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.760894    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4685e84e-edb6-4ea1-b396-db534f083bcd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.760975 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.760922    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4685e84e-edb6-4ea1-b396-db534f083bcd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.761085 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.761001    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4685e84e-edb6-4ea1-b396-db534f083bcd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.763001 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.762980    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4685e84e-edb6-4ea1-b396-db534f083bcd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.763291 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.763273    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4685e84e-edb6-4ea1-b396-db534f083bcd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.769296 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.769266    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m552s\" (UniqueName: \"kubernetes.io/projected/4685e84e-edb6-4ea1-b396-db534f083bcd-kube-api-access-m552s\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hzhtk\" (UID: \"4685e84e-edb6-4ea1-b396-db534f083bcd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.822672 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.822628    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:22:51.962581 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:51.962550    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"]
Jul 09 12:22:51.964122 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:22:51.964091    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4685e84e_edb6_4ea1_b396_db534f083bcd.slice/crio-8951888dc79cfe457af6d12f02bb6e7e36a03e3be6c8b94dbdcaea04c2f5d947 WatchSource:0}: Error finding container 8951888dc79cfe457af6d12f02bb6e7e36a03e3be6c8b94dbdcaea04c2f5d947: Status 404 returned error can't find the container with id 8951888dc79cfe457af6d12f02bb6e7e36a03e3be6c8b94dbdcaea04c2f5d947
Jul 09 12:22:52.051734 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:52.051614    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk" event={"ID":"4685e84e-edb6-4ea1-b396-db534f083bcd","Type":"ContainerStarted","Data":"8951888dc79cfe457af6d12f02bb6e7e36a03e3be6c8b94dbdcaea04c2f5d947"}
Jul 09 12:22:53.058716 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:53.058670    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk" event={"ID":"4685e84e-edb6-4ea1-b396-db534f083bcd","Type":"ContainerStarted","Data":"9fa22dc1117047778a6699e071f900d0c217a1699804a0882bf2222b84bb10dd"}
Jul 09 12:22:53.079952 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:53.079892    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk" podStartSLOduration=1.859586052 podStartE2EDuration="2.079873566s" totalImagesPullingTime="220.287514ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:22:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:22:53.077590272 +0000 UTC m=+1245.755119331" watchObservedRunningTime="2026-07-09 12:22:53.079873566 +0000 UTC m=+1245.757402625"
Jul 09 12:22:53.118326 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:22:53.118291    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:23:01.823558 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:01.823463    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:23:01.823997 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:01.823569    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:23:01.836135 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:01.836106    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:23:02.116236 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:02.116140    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hzhtk"
Jul 09 12:23:08.729460 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:08.729424    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"]
Jul 09 12:23:08.732939 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:08.732919    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:08.736045 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:08.736020    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 09 12:23:08.743415 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:08.743387    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"]
Jul 09 12:23:08.919032 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:08.918992    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nlwr\" (UniqueName: \"kubernetes.io/projected/9255360e-c0fe-48a2-b920-421af01ceccc-kube-api-access-2nlwr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:08.919220 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:08.919049    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9255360e-c0fe-48a2-b920-421af01ceccc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:08.919220 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:08.919073    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9255360e-c0fe-48a2-b920-421af01ceccc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:08.919220 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:08.919100    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9255360e-c0fe-48a2-b920-421af01ceccc-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:08.919220 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:08.919114    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9255360e-c0fe-48a2-b920-421af01ceccc-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:08.919357 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:08.919215    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9255360e-c0fe-48a2-b920-421af01ceccc-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:09.020352 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:09.020254    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9255360e-c0fe-48a2-b920-421af01ceccc-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:09.020352 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:09.020294    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9255360e-c0fe-48a2-b920-421af01ceccc-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:09.020352 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:09.020347    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9255360e-c0fe-48a2-b920-421af01ceccc-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:09.020630 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:09.020401    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nlwr\" (UniqueName: \"kubernetes.io/projected/9255360e-c0fe-48a2-b920-421af01ceccc-kube-api-access-2nlwr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:09.020630 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:09.020448    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9255360e-c0fe-48a2-b920-421af01ceccc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:09.020630 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:09.020495    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9255360e-c0fe-48a2-b920-421af01ceccc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:09.020829 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:09.020753    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9255360e-c0fe-48a2-b920-421af01ceccc-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:09.020829 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:09.020801    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9255360e-c0fe-48a2-b920-421af01ceccc-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:09.020829 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:09.020826    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9255360e-c0fe-48a2-b920-421af01ceccc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:09.022883 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:09.022843    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9255360e-c0fe-48a2-b920-421af01ceccc-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:09.023080 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:09.023060    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9255360e-c0fe-48a2-b920-421af01ceccc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:09.029451 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:09.029425    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nlwr\" (UniqueName: \"kubernetes.io/projected/9255360e-c0fe-48a2-b920-421af01ceccc-kube-api-access-2nlwr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg\" (UID: \"9255360e-c0fe-48a2-b920-421af01ceccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:09.042343 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:09.042312    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:09.182654 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:09.182619    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"]
Jul 09 12:23:09.183963 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:23:09.183929    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9255360e_c0fe_48a2_b920_421af01ceccc.slice/crio-3d61969bcaa3d7da6da7673e6c77c6c4d9a9938d443c6b4ae13972027bc7232b WatchSource:0}: Error finding container 3d61969bcaa3d7da6da7673e6c77c6c4d9a9938d443c6b4ae13972027bc7232b: Status 404 returned error can't find the container with id 3d61969bcaa3d7da6da7673e6c77c6c4d9a9938d443c6b4ae13972027bc7232b
Jul 09 12:23:10.155124 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:10.155091    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg" event={"ID":"9255360e-c0fe-48a2-b920-421af01ceccc","Type":"ContainerStarted","Data":"0b59abc36dff85b1ca70a8515785edce5a1c5c93026018ed9a84ce9543e199b2"}
Jul 09 12:23:10.155515 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:10.155134    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg" event={"ID":"9255360e-c0fe-48a2-b920-421af01ceccc","Type":"ContainerStarted","Data":"3d61969bcaa3d7da6da7673e6c77c6c4d9a9938d443c6b4ae13972027bc7232b"}
Jul 09 12:23:10.174425 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:10.174370    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg" podStartSLOduration=1.894300742 podStartE2EDuration="2.174353274s" totalImagesPullingTime="280.052532ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:23:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:23:10.171963407 +0000 UTC m=+1262.849492466" watchObservedRunningTime="2026-07-09 12:23:10.174353274 +0000 UTC m=+1262.851882348"
Jul 09 12:23:11.338927 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:11.338893    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:23:19.043246 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:19.043201    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:19.043246 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:19.043255    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:19.056471 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:19.056438    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:23:19.213763 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:23:19.213733    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg"
Jul 09 12:24:30.622299 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:24:30.622202    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:24:39.630298 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:24:39.630264    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:24:47.023091 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:24:47.023052    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:24:56.720595 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:24:56.720554    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:25:06.724090 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:25:06.724048    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:25:14.821893 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:25:14.821859    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:25:38.825873 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:25:38.825829    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:25:51.425061 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:25:51.425023    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:26:27.826097 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:26:27.825981    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:26:37.324565 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:26:37.324516    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:26:46.425536 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:26:46.425495    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:26:52.534759 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:26:52.534717    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:27:03.451064 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:27:03.451030    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:27:07.909646 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:27:07.909615    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:27:07.910962 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:27:07.910944    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:27:12.822399 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:27:12.822364    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:27:23.650103 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:27:23.650056    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:27:33.034508 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:27:33.034422    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:27:55.820039 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:27:55.819998    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:28:07.628464 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:28:07.628421    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:28:31.633184 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:28:31.633148    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:28:43.124307 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:28:43.124271    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:28:56.735551 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:28:56.735510    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:29:08.028105 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:29:08.028021    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:29:17.627667 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:29:17.627626    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:29:27.516434 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:29:27.516396    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:29:37.126631 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:29:37.126588    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:31:22.021860 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:31:22.021828    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:31:34.131007 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:31:34.130965    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:31:43.026270 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:31:43.026234    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:31:54.466507 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:31:54.466464    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:32:06.119683 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:32:06.119647    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:32:07.949998 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:32:07.949965    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:32:07.952105 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:32:07.952082    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:32:16.424330 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:32:16.424291    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:32:25.222839 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:32:25.222798    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:32:36.720251 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:32:36.720215    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:32:47.225197 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:32:47.225153    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:32:55.618943 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:32:55.618902    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:33:06.029179 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:33:06.029144    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:33:28.218685 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:33:28.218642    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:33:37.428712 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:33:37.428669    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:33:47.025349 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:33:47.025311    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:33:58.520963 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:33:58.520919    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:34:13.326212 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:34:13.326169    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:34:23.218676 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:34:23.218626    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:34:32.929282 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:34:32.929240    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:34:43.424804 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:34:43.424762    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:34:52.530570 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:34:52.530526    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:35:02.516628 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:35:02.516530    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:35:23.722386 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:35:23.722344    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:35:33.417837 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:35:33.417800    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:35:43.119623 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:35:43.119586    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:35:56.820204 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:35:56.820162    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:36:05.742391 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:36:05.742346    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:36:18.129653 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:36:18.129593    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:36:30.424442 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:36:30.424359    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:36:42.918197 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:36:42.918158    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:36:53.323600 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:36:53.323561    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:37:07.985964 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:07.985938    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:37:07.989590 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:07.989566    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:37:14.331174 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:14.331136    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:37:20.489100 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.489058    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"]
Jul 09 12:37:20.492643 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.492622    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.495431 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.495403    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-bf66c10b-openshift-default-dockercfg-l659z\""
Jul 09 12:37:20.511275 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.511251    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"]
Jul 09 12:37:20.652562 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.652514    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-workload-certs\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.652562 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.652563    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-podinfo\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.652835 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.652580    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-token\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.652835 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.652631    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-data\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.652835 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.652675    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-envoy\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.652835 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.652753    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istiod-ca-cert\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.652835 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.652790    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-workload-socket\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.652835 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.652814    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-credential-socket\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.653025 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.652854    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzhx7\" (UniqueName: \"kubernetes.io/projected/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-kube-api-access-jzhx7\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.753995 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.753893    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-workload-certs\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.753995 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.753937    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-podinfo\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.753995 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.753955    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-token\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.754296 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.754151    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-data\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.754296 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.754200    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-envoy\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.754296 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.754277    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istiod-ca-cert\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.754463 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.754336    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-workload-socket\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.754463 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.754394    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-credential-socket\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.754463 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.754435    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzhx7\" (UniqueName: \"kubernetes.io/projected/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-kube-api-access-jzhx7\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.754606 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.754487    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-workload-certs\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.754651 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.754613    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-data\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.754809 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.754784    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-credential-socket\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.755063 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.755039    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istiod-ca-cert\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.755135 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.755074    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-workload-socket\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.756467 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.756438    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-envoy\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.756827 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.756806    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-podinfo\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.763674 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.763643    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-token\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.763880 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.763862    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzhx7\" (UniqueName: \"kubernetes.io/projected/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-kube-api-access-jzhx7\") pod \"e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") " pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.803714 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.803677    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:20.953907 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.953871    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"]
Jul 09 12:37:20.955612 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:37:20.955588    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d9b8809_0df5_4e1d_bb9c_7cc15716c4c6.slice/crio-e3dd2b235116150c1d0c1fab740a79ea4ecbac5f69a83b5f7d05223d66f6b485 WatchSource:0}: Error finding container e3dd2b235116150c1d0c1fab740a79ea4ecbac5f69a83b5f7d05223d66f6b485: Status 404 returned error can't find the container with id e3dd2b235116150c1d0c1fab740a79ea4ecbac5f69a83b5f7d05223d66f6b485
Jul 09 12:37:20.957716 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.957670    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:37:20.957807 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.957760    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:37:20.957807 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:20.957794    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:37:21.762268 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.762236    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"]
Jul 09 12:37:21.765923 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.765901    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:21.768598 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.768573    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bf66c10b-serving-cert\""
Jul 09 12:37:21.777061 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.777035    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"]
Jul 09 12:37:21.809387 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.809349    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65" event={"ID":"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6","Type":"ContainerStarted","Data":"35b611650ee9b1b4f1e3a352161c9f13213d9a13e658ae546fa1ba6d3cb660aa"}
Jul 09 12:37:21.809570 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.809407    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65" event={"ID":"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6","Type":"ContainerStarted","Data":"e3dd2b235116150c1d0c1fab740a79ea4ecbac5f69a83b5f7d05223d66f6b485"}
Jul 09 12:37:21.865558 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.865520    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr28q\" (UniqueName: \"kubernetes.io/projected/70c84c31-cd18-4184-bd6f-dd26e34153a5-kube-api-access-lr28q\") pod \"maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm\" (UID: \"70c84c31-cd18-4184-bd6f-dd26e34153a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:21.865784 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.865650    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c84c31-cd18-4184-bd6f-dd26e34153a5-openshift-service-ca\") pod \"maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm\" (UID: \"70c84c31-cd18-4184-bd6f-dd26e34153a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:21.865784 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.865740    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c84c31-cd18-4184-bd6f-dd26e34153a5-maas-api-tls\") pod \"maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm\" (UID: \"70c84c31-cd18-4184-bd6f-dd26e34153a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:21.868481 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.868425    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65" podStartSLOduration=1.868399625 podStartE2EDuration="1.868399625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:37:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:37:21.862865298 +0000 UTC m=+2114.540394357" watchObservedRunningTime="2026-07-09 12:37:21.868399625 +0000 UTC m=+2114.545928684"
Jul 09 12:37:21.966910 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.966864    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c84c31-cd18-4184-bd6f-dd26e34153a5-openshift-service-ca\") pod \"maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm\" (UID: \"70c84c31-cd18-4184-bd6f-dd26e34153a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:21.967107 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.966955    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c84c31-cd18-4184-bd6f-dd26e34153a5-maas-api-tls\") pod \"maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm\" (UID: \"70c84c31-cd18-4184-bd6f-dd26e34153a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:21.967107 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.967004    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr28q\" (UniqueName: \"kubernetes.io/projected/70c84c31-cd18-4184-bd6f-dd26e34153a5-kube-api-access-lr28q\") pod \"maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm\" (UID: \"70c84c31-cd18-4184-bd6f-dd26e34153a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:21.967233 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:37:21.967105    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-serving-cert: secret "maas-api-e2e-ait-bf66c10b-serving-cert" not found
Jul 09 12:37:21.967233 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:37:21.967177    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70c84c31-cd18-4184-bd6f-dd26e34153a5-maas-api-tls podName:70c84c31-cd18-4184-bd6f-dd26e34153a5 nodeName:}" failed. No retries permitted until 2026-07-09 12:37:22.467153614 +0000 UTC m=+2115.144682673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/70c84c31-cd18-4184-bd6f-dd26e34153a5-maas-api-tls") pod "maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm" (UID: "70c84c31-cd18-4184-bd6f-dd26e34153a5") : secret "maas-api-e2e-ait-bf66c10b-serving-cert" not found
Jul 09 12:37:21.967654 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.967632    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c84c31-cd18-4184-bd6f-dd26e34153a5-openshift-service-ca\") pod \"maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm\" (UID: \"70c84c31-cd18-4184-bd6f-dd26e34153a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:21.985890 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:21.985853    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr28q\" (UniqueName: \"kubernetes.io/projected/70c84c31-cd18-4184-bd6f-dd26e34153a5-kube-api-access-lr28q\") pod \"maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm\" (UID: \"70c84c31-cd18-4184-bd6f-dd26e34153a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:22.471550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:22.471512    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c84c31-cd18-4184-bd6f-dd26e34153a5-maas-api-tls\") pod \"maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm\" (UID: \"70c84c31-cd18-4184-bd6f-dd26e34153a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:22.474661 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:22.474628    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c84c31-cd18-4184-bd6f-dd26e34153a5-maas-api-tls\") pod \"maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm\" (UID: \"70c84c31-cd18-4184-bd6f-dd26e34153a5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:22.679578 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:22.679544    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:22.804379 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:22.804334    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:22.809927 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:22.809895    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:22.815531 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:22.815490    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:22.816517 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:22.816497    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:22.823641 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:22.823621    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"]
Jul 09 12:37:22.825392 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:37:22.825369    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c84c31_cd18_4184_bd6f_dd26e34153a5.slice/crio-8bde67a55f8a0fb5719d73a5ee540193666b8348cfa9ffc1dda079c0ca7ede64 WatchSource:0}: Error finding container 8bde67a55f8a0fb5719d73a5ee540193666b8348cfa9ffc1dda079c0ca7ede64: Status 404 returned error can't find the container with id 8bde67a55f8a0fb5719d73a5ee540193666b8348cfa9ffc1dda079c0ca7ede64
Jul 09 12:37:22.826716 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:22.826686    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 12:37:23.822924 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:23.822885    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm" event={"ID":"70c84c31-cd18-4184-bd6f-dd26e34153a5","Type":"ContainerStarted","Data":"ac343851c45c01988df2a9ece0f3153ccdc557ba6072e0dec36cb2b4c8e3c8c7"}
Jul 09 12:37:23.823411 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:23.822941    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm" event={"ID":"70c84c31-cd18-4184-bd6f-dd26e34153a5","Type":"ContainerStarted","Data":"8bde67a55f8a0fb5719d73a5ee540193666b8348cfa9ffc1dda079c0ca7ede64"}
Jul 09 12:37:23.823411 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:23.822963    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:23.840255 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:23.840189    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm" podStartSLOduration=2.283831987 podStartE2EDuration="2.840169218s" totalImagesPullingTime="556.337231ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:37:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:37:23.839787345 +0000 UTC m=+2116.517316406" watchObservedRunningTime="2026-07-09 12:37:23.840169218 +0000 UTC m=+2116.517698278"
Jul 09 12:37:25.230178 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:25.230140    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:37:26.865692 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:26.865650    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"]
Jul 09 12:37:26.866160 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:26.865877    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65" podUID="3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" containerName="istio-proxy" containerID="cri-o://35b611650ee9b1b4f1e3a352161c9f13213d9a13e658ae546fa1ba6d3cb660aa" gracePeriod=30
Jul 09 12:37:27.046879 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.046841    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"]
Jul 09 12:37:27.047350 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.047316    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm" podUID="70c84c31-cd18-4184-bd6f-dd26e34153a5" containerName="maas-api" containerID="cri-o://ac343851c45c01988df2a9ece0f3153ccdc557ba6072e0dec36cb2b4c8e3c8c7" gracePeriod=30
Jul 09 12:37:27.344005 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.343967    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:27.441253 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.441158    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lr28q\" (UniqueName: \"kubernetes.io/projected/70c84c31-cd18-4184-bd6f-dd26e34153a5-kube-api-access-lr28q\") pod \"70c84c31-cd18-4184-bd6f-dd26e34153a5\" (UID: \"70c84c31-cd18-4184-bd6f-dd26e34153a5\") "
Jul 09 12:37:27.441253 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.441216    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c84c31-cd18-4184-bd6f-dd26e34153a5-maas-api-tls\") pod \"70c84c31-cd18-4184-bd6f-dd26e34153a5\" (UID: \"70c84c31-cd18-4184-bd6f-dd26e34153a5\") "
Jul 09 12:37:27.441493 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.441326    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c84c31-cd18-4184-bd6f-dd26e34153a5-openshift-service-ca\") pod \"70c84c31-cd18-4184-bd6f-dd26e34153a5\" (UID: \"70c84c31-cd18-4184-bd6f-dd26e34153a5\") "
Jul 09 12:37:27.441694 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.441666    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c84c31-cd18-4184-bd6f-dd26e34153a5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70c84c31-cd18-4184-bd6f-dd26e34153a5" (UID: "70c84c31-cd18-4184-bd6f-dd26e34153a5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:37:27.444216 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.444187    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c84c31-cd18-4184-bd6f-dd26e34153a5-kube-api-access-lr28q" (OuterVolumeSpecName: "kube-api-access-lr28q") pod "70c84c31-cd18-4184-bd6f-dd26e34153a5" (UID: "70c84c31-cd18-4184-bd6f-dd26e34153a5"). InnerVolumeSpecName "kube-api-access-lr28q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:37:27.444216 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.444188    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c84c31-cd18-4184-bd6f-dd26e34153a5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70c84c31-cd18-4184-bd6f-dd26e34153a5" (UID: "70c84c31-cd18-4184-bd6f-dd26e34153a5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:37:27.542834 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.542795    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lr28q\" (UniqueName: \"kubernetes.io/projected/70c84c31-cd18-4184-bd6f-dd26e34153a5-kube-api-access-lr28q\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:27.542834 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.542828    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70c84c31-cd18-4184-bd6f-dd26e34153a5-maas-api-tls\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:27.542834 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.542838    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c84c31-cd18-4184-bd6f-dd26e34153a5-openshift-service-ca\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:27.849887 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.849849    2565 generic.go:356] "Generic (PLEG): container finished" podID="70c84c31-cd18-4184-bd6f-dd26e34153a5" containerID="ac343851c45c01988df2a9ece0f3153ccdc557ba6072e0dec36cb2b4c8e3c8c7" exitCode=0
Jul 09 12:37:27.850039 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.849898    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"
Jul 09 12:37:27.850039 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.849937    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm" event={"ID":"70c84c31-cd18-4184-bd6f-dd26e34153a5","Type":"ContainerDied","Data":"ac343851c45c01988df2a9ece0f3153ccdc557ba6072e0dec36cb2b4c8e3c8c7"}
Jul 09 12:37:27.850039 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.849999    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm" event={"ID":"70c84c31-cd18-4184-bd6f-dd26e34153a5","Type":"ContainerDied","Data":"8bde67a55f8a0fb5719d73a5ee540193666b8348cfa9ffc1dda079c0ca7ede64"}
Jul 09 12:37:27.850039 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.850024    2565 scope.go:117] "RemoveContainer" containerID="ac343851c45c01988df2a9ece0f3153ccdc557ba6072e0dec36cb2b4c8e3c8c7"
Jul 09 12:37:27.861578 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.861553    2565 scope.go:117] "RemoveContainer" containerID="ac343851c45c01988df2a9ece0f3153ccdc557ba6072e0dec36cb2b4c8e3c8c7"
Jul 09 12:37:27.861964 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:37:27.861948    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac343851c45c01988df2a9ece0f3153ccdc557ba6072e0dec36cb2b4c8e3c8c7\": container with ID starting with ac343851c45c01988df2a9ece0f3153ccdc557ba6072e0dec36cb2b4c8e3c8c7 not found: ID does not exist" containerID="ac343851c45c01988df2a9ece0f3153ccdc557ba6072e0dec36cb2b4c8e3c8c7"
Jul 09 12:37:27.862030 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.861972    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac343851c45c01988df2a9ece0f3153ccdc557ba6072e0dec36cb2b4c8e3c8c7"} err="failed to get container status \"ac343851c45c01988df2a9ece0f3153ccdc557ba6072e0dec36cb2b4c8e3c8c7\": rpc error: code = NotFound desc = could not find container \"ac343851c45c01988df2a9ece0f3153ccdc557ba6072e0dec36cb2b4c8e3c8c7\": container with ID starting with ac343851c45c01988df2a9ece0f3153ccdc557ba6072e0dec36cb2b4c8e3c8c7 not found: ID does not exist"
Jul 09 12:37:27.874593 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.874560    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"]
Jul 09 12:37:27.878122 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.878095    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bf66c10b-56bf548995-qlbbm"]
Jul 09 12:37:27.899980 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:27.899942    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70c84c31-cd18-4184-bd6f-dd26e34153a5" path="/var/lib/kubelet/pods/70c84c31-cd18-4184-bd6f-dd26e34153a5/volumes"
Jul 09 12:37:33.727963 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.727900    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:33.806244 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.806209    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-workload-socket\") pod \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") "
Jul 09 12:37:33.806244 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.806249    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-envoy\") pod \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") "
Jul 09 12:37:33.806516 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.806266    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-credential-socket\") pod \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") "
Jul 09 12:37:33.806516 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.806286    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-podinfo\") pod \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") "
Jul 09 12:37:33.806516 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.806303    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzhx7\" (UniqueName: \"kubernetes.io/projected/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-kube-api-access-jzhx7\") pod \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") "
Jul 09 12:37:33.806516 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.806346    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-token\") pod \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") "
Jul 09 12:37:33.806516 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.806363    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-data\") pod \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") "
Jul 09 12:37:33.806516 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.806404    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-workload-certs\") pod \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") "
Jul 09 12:37:33.806516 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.806427    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istiod-ca-cert\") pod \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\" (UID: \"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6\") "
Jul 09 12:37:33.806858 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.806823    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" (UID: "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:37:33.806970 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.806945    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" (UID: "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:37:33.807148 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.807119    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" (UID: "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:37:33.807745 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.807689    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" (UID: "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:37:33.807996 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.807927    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-data" (OuterVolumeSpecName: "istio-data") pod "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" (UID: "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:37:33.808502 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.808474    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-workload-socket\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:33.808600 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.808507    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-credential-socket\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:33.808600 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.808531    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-data\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:33.808600 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.808545    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-workload-certs\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:33.808600 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.808557    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istiod-ca-cert\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:33.810185 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.810117    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" (UID: "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 12:37:33.816059 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.816016    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" (UID: "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:37:33.816197 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.816057    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-kube-api-access-jzhx7" (OuterVolumeSpecName: "kube-api-access-jzhx7") pod "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" (UID: "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6"). InnerVolumeSpecName "kube-api-access-jzhx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:37:33.816197 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.816057    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" (UID: "3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:37:33.886259 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.886220    2565 generic.go:356] "Generic (PLEG): container finished" podID="3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" containerID="35b611650ee9b1b4f1e3a352161c9f13213d9a13e658ae546fa1ba6d3cb660aa" exitCode=0
Jul 09 12:37:33.886409 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.886289    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"
Jul 09 12:37:33.886409 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.886302    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65" event={"ID":"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6","Type":"ContainerDied","Data":"35b611650ee9b1b4f1e3a352161c9f13213d9a13e658ae546fa1ba6d3cb660aa"}
Jul 09 12:37:33.886409 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.886393    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65" event={"ID":"3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6","Type":"ContainerDied","Data":"e3dd2b235116150c1d0c1fab740a79ea4ecbac5f69a83b5f7d05223d66f6b485"}
Jul 09 12:37:33.886550 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.886417    2565 scope.go:117] "RemoveContainer" containerID="35b611650ee9b1b4f1e3a352161c9f13213d9a13e658ae546fa1ba6d3cb660aa"
Jul 09 12:37:33.897148 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.897119    2565 scope.go:117] "RemoveContainer" containerID="35b611650ee9b1b4f1e3a352161c9f13213d9a13e658ae546fa1ba6d3cb660aa"
Jul 09 12:37:33.897461 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:37:33.897440    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35b611650ee9b1b4f1e3a352161c9f13213d9a13e658ae546fa1ba6d3cb660aa\": container with ID starting with 35b611650ee9b1b4f1e3a352161c9f13213d9a13e658ae546fa1ba6d3cb660aa not found: ID does not exist" containerID="35b611650ee9b1b4f1e3a352161c9f13213d9a13e658ae546fa1ba6d3cb660aa"
Jul 09 12:37:33.897509 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.897472    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35b611650ee9b1b4f1e3a352161c9f13213d9a13e658ae546fa1ba6d3cb660aa"} err="failed to get container status \"35b611650ee9b1b4f1e3a352161c9f13213d9a13e658ae546fa1ba6d3cb660aa\": rpc error: code = NotFound desc = could not find container \"35b611650ee9b1b4f1e3a352161c9f13213d9a13e658ae546fa1ba6d3cb660aa\": container with ID starting with 35b611650ee9b1b4f1e3a352161c9f13213d9a13e658ae546fa1ba6d3cb660aa not found: ID does not exist"
Jul 09 12:37:33.909140 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.909106    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-envoy\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:33.909140 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.909141    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-podinfo\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:33.909348 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.909154    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzhx7\" (UniqueName: \"kubernetes.io/projected/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-kube-api-access-jzhx7\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:33.909348 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.909168    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6-istio-token\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:33.911291 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.911265    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"]
Jul 09 12:37:33.915093 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:33.915062    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-bf66c10b-openshift-default-5cbf4b7699-cjq65"]
Jul 09 12:37:34.273590 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.273554    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"]
Jul 09 12:37:34.274008 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.273990    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c84c31-cd18-4184-bd6f-dd26e34153a5" containerName="maas-api"
Jul 09 12:37:34.274008 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.274009    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c84c31-cd18-4184-bd6f-dd26e34153a5" containerName="maas-api"
Jul 09 12:37:34.274143 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.274023    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" containerName="istio-proxy"
Jul 09 12:37:34.274143 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.274029    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" containerName="istio-proxy"
Jul 09 12:37:34.274143 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.274102    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" containerName="istio-proxy"
Jul 09 12:37:34.274143 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.274114    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c84c31-cd18-4184-bd6f-dd26e34153a5" containerName="maas-api"
Jul 09 12:37:34.277488 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.277466    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:34.280446 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.280426    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f774bd66-serving-cert\""
Jul 09 12:37:34.293931 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.293895    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"]
Jul 09 12:37:34.313973 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.313938    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7aab477-6409-4fc9-9cae-267c33b30a19-maas-api-tls\") pod \"maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5\" (UID: \"c7aab477-6409-4fc9-9cae-267c33b30a19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:34.314155 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.313980    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7aab477-6409-4fc9-9cae-267c33b30a19-openshift-service-ca\") pod \"maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5\" (UID: \"c7aab477-6409-4fc9-9cae-267c33b30a19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:34.314155 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.314107    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp8fk\" (UniqueName: \"kubernetes.io/projected/c7aab477-6409-4fc9-9cae-267c33b30a19-kube-api-access-wp8fk\") pod \"maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5\" (UID: \"c7aab477-6409-4fc9-9cae-267c33b30a19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:34.414689 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.414649    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7aab477-6409-4fc9-9cae-267c33b30a19-maas-api-tls\") pod \"maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5\" (UID: \"c7aab477-6409-4fc9-9cae-267c33b30a19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:34.414689 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.414693    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7aab477-6409-4fc9-9cae-267c33b30a19-openshift-service-ca\") pod \"maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5\" (UID: \"c7aab477-6409-4fc9-9cae-267c33b30a19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:34.414976 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.414753    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp8fk\" (UniqueName: \"kubernetes.io/projected/c7aab477-6409-4fc9-9cae-267c33b30a19-kube-api-access-wp8fk\") pod \"maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5\" (UID: \"c7aab477-6409-4fc9-9cae-267c33b30a19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:34.414976 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:37:34.414821    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-serving-cert: secret "maas-api-e2e-ait-f774bd66-serving-cert" not found
Jul 09 12:37:34.414976 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:37:34.414902    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7aab477-6409-4fc9-9cae-267c33b30a19-maas-api-tls podName:c7aab477-6409-4fc9-9cae-267c33b30a19 nodeName:}" failed. No retries permitted until 2026-07-09 12:37:34.914884229 +0000 UTC m=+2127.592413264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c7aab477-6409-4fc9-9cae-267c33b30a19-maas-api-tls") pod "maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5" (UID: "c7aab477-6409-4fc9-9cae-267c33b30a19") : secret "maas-api-e2e-ait-f774bd66-serving-cert" not found
Jul 09 12:37:34.415669 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.415643    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7aab477-6409-4fc9-9cae-267c33b30a19-openshift-service-ca\") pod \"maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5\" (UID: \"c7aab477-6409-4fc9-9cae-267c33b30a19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:34.429064 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.429025    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp8fk\" (UniqueName: \"kubernetes.io/projected/c7aab477-6409-4fc9-9cae-267c33b30a19-kube-api-access-wp8fk\") pod \"maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5\" (UID: \"c7aab477-6409-4fc9-9cae-267c33b30a19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:34.919491 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.919455    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7aab477-6409-4fc9-9cae-267c33b30a19-maas-api-tls\") pod \"maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5\" (UID: \"c7aab477-6409-4fc9-9cae-267c33b30a19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:34.922364 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:34.922334    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7aab477-6409-4fc9-9cae-267c33b30a19-maas-api-tls\") pod \"maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5\" (UID: \"c7aab477-6409-4fc9-9cae-267c33b30a19\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:35.188310 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:35.188207    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:35.344810 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:35.344782    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"]
Jul 09 12:37:35.346507 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:37:35.346479    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7aab477_6409_4fc9_9cae_267c33b30a19.slice/crio-11596d171876e24f1da523f51a9984139bea9b8526f6eaec8cc5dcf3c05cdf6a WatchSource:0}: Error finding container 11596d171876e24f1da523f51a9984139bea9b8526f6eaec8cc5dcf3c05cdf6a: Status 404 returned error can't find the container with id 11596d171876e24f1da523f51a9984139bea9b8526f6eaec8cc5dcf3c05cdf6a
Jul 09 12:37:35.894615 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:35.894585    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6" path="/var/lib/kubelet/pods/3d9b8809-0df5-4e1d-bb9c-7cc15716c4c6/volumes"
Jul 09 12:37:35.901099 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:35.901069    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5" event={"ID":"c7aab477-6409-4fc9-9cae-267c33b30a19","Type":"ContainerStarted","Data":"f1f374e0ac7bc11fbf8ae3dcf12df9b21db99dd6dd18fc11eca2c44ef6d98742"}
Jul 09 12:37:35.901260 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:35.901120    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5" event={"ID":"c7aab477-6409-4fc9-9cae-267c33b30a19","Type":"ContainerStarted","Data":"11596d171876e24f1da523f51a9984139bea9b8526f6eaec8cc5dcf3c05cdf6a"}
Jul 09 12:37:35.901260 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:35.901146    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:35.918892 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:35.918833    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5" podStartSLOduration=1.588859383 podStartE2EDuration="1.918812515s" totalImagesPullingTime="329.953132ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:37:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:37:35.915977999 +0000 UTC m=+2128.593507072" watchObservedRunningTime="2026-07-09 12:37:35.918812515 +0000 UTC m=+2128.596341576"
Jul 09 12:37:37.020189 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:37.020154    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:37:39.879488 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:39.879450    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"]
Jul 09 12:37:39.880043 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:39.879742    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5" podUID="c7aab477-6409-4fc9-9cae-267c33b30a19" containerName="maas-api" containerID="cri-o://f1f374e0ac7bc11fbf8ae3dcf12df9b21db99dd6dd18fc11eca2c44ef6d98742" gracePeriod=30
Jul 09 12:37:40.134456 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.134388    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:40.168407 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.168373    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7aab477-6409-4fc9-9cae-267c33b30a19-openshift-service-ca\") pod \"c7aab477-6409-4fc9-9cae-267c33b30a19\" (UID: \"c7aab477-6409-4fc9-9cae-267c33b30a19\") "
Jul 09 12:37:40.168616 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.168419    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7aab477-6409-4fc9-9cae-267c33b30a19-maas-api-tls\") pod \"c7aab477-6409-4fc9-9cae-267c33b30a19\" (UID: \"c7aab477-6409-4fc9-9cae-267c33b30a19\") "
Jul 09 12:37:40.168616 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.168549    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wp8fk\" (UniqueName: \"kubernetes.io/projected/c7aab477-6409-4fc9-9cae-267c33b30a19-kube-api-access-wp8fk\") pod \"c7aab477-6409-4fc9-9cae-267c33b30a19\" (UID: \"c7aab477-6409-4fc9-9cae-267c33b30a19\") "
Jul 09 12:37:40.168821 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.168793    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7aab477-6409-4fc9-9cae-267c33b30a19-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c7aab477-6409-4fc9-9cae-267c33b30a19" (UID: "c7aab477-6409-4fc9-9cae-267c33b30a19"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:37:40.168913 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.168894    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7aab477-6409-4fc9-9cae-267c33b30a19-openshift-service-ca\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:40.170832 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.170801    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7aab477-6409-4fc9-9cae-267c33b30a19-kube-api-access-wp8fk" (OuterVolumeSpecName: "kube-api-access-wp8fk") pod "c7aab477-6409-4fc9-9cae-267c33b30a19" (UID: "c7aab477-6409-4fc9-9cae-267c33b30a19"). InnerVolumeSpecName "kube-api-access-wp8fk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:37:40.170832 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.170813    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7aab477-6409-4fc9-9cae-267c33b30a19-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c7aab477-6409-4fc9-9cae-267c33b30a19" (UID: "c7aab477-6409-4fc9-9cae-267c33b30a19"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:37:40.269727 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.269657    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp8fk\" (UniqueName: \"kubernetes.io/projected/c7aab477-6409-4fc9-9cae-267c33b30a19-kube-api-access-wp8fk\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:40.269727 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.269692    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7aab477-6409-4fc9-9cae-267c33b30a19-maas-api-tls\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:40.928382 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.928352    2565 generic.go:356] "Generic (PLEG): container finished" podID="c7aab477-6409-4fc9-9cae-267c33b30a19" containerID="f1f374e0ac7bc11fbf8ae3dcf12df9b21db99dd6dd18fc11eca2c44ef6d98742" exitCode=0
Jul 09 12:37:40.928910 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.928395    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"
Jul 09 12:37:40.928910 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.928442    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5" event={"ID":"c7aab477-6409-4fc9-9cae-267c33b30a19","Type":"ContainerDied","Data":"f1f374e0ac7bc11fbf8ae3dcf12df9b21db99dd6dd18fc11eca2c44ef6d98742"}
Jul 09 12:37:40.928910 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.928511    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5" event={"ID":"c7aab477-6409-4fc9-9cae-267c33b30a19","Type":"ContainerDied","Data":"11596d171876e24f1da523f51a9984139bea9b8526f6eaec8cc5dcf3c05cdf6a"}
Jul 09 12:37:40.928910 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.928547    2565 scope.go:117] "RemoveContainer" containerID="f1f374e0ac7bc11fbf8ae3dcf12df9b21db99dd6dd18fc11eca2c44ef6d98742"
Jul 09 12:37:40.940645 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.940625    2565 scope.go:117] "RemoveContainer" containerID="f1f374e0ac7bc11fbf8ae3dcf12df9b21db99dd6dd18fc11eca2c44ef6d98742"
Jul 09 12:37:40.940994 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:37:40.940975    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1f374e0ac7bc11fbf8ae3dcf12df9b21db99dd6dd18fc11eca2c44ef6d98742\": container with ID starting with f1f374e0ac7bc11fbf8ae3dcf12df9b21db99dd6dd18fc11eca2c44ef6d98742 not found: ID does not exist" containerID="f1f374e0ac7bc11fbf8ae3dcf12df9b21db99dd6dd18fc11eca2c44ef6d98742"
Jul 09 12:37:40.941049 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.941003    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1f374e0ac7bc11fbf8ae3dcf12df9b21db99dd6dd18fc11eca2c44ef6d98742"} err="failed to get container status \"f1f374e0ac7bc11fbf8ae3dcf12df9b21db99dd6dd18fc11eca2c44ef6d98742\": rpc error: code = NotFound desc = could not find container \"f1f374e0ac7bc11fbf8ae3dcf12df9b21db99dd6dd18fc11eca2c44ef6d98742\": container with ID starting with f1f374e0ac7bc11fbf8ae3dcf12df9b21db99dd6dd18fc11eca2c44ef6d98742 not found: ID does not exist"
Jul 09 12:37:40.955202 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.955171    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"]
Jul 09 12:37:40.959120 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:40.959093    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f774bd66-89bbc44b7-jxtz5"]
Jul 09 12:37:41.894723 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:41.894677    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7aab477-6409-4fc9-9cae-267c33b30a19" path="/var/lib/kubelet/pods/c7aab477-6409-4fc9-9cae-267c33b30a19/volumes"
Jul 09 12:37:48.913967 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:48.913926    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:37:51.933044 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:51.933016    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"]
Jul 09 12:37:51.933403 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:51.933389    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7aab477-6409-4fc9-9cae-267c33b30a19" containerName="maas-api"
Jul 09 12:37:51.933403 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:51.933403    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7aab477-6409-4fc9-9cae-267c33b30a19" containerName="maas-api"
Jul 09 12:37:51.933489 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:51.933470    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7aab477-6409-4fc9-9cae-267c33b30a19" containerName="maas-api"
Jul 09 12:37:51.935805 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:51.935787    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:51.938564 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:51.938541    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9b6bd36b-openshift-default-dockercfg-9qsk7\""
Jul 09 12:37:51.948300 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:51.948276    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"]
Jul 09 12:37:52.082385 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.082347    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-workload-certs\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.082385 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.082398    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-credential-socket\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.082656 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.082488    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-envoy\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.082656 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.082534    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-token\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.082656 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.082577    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9khd\" (UniqueName: \"kubernetes.io/projected/90b8fc0b-5847-4799-adb5-141cfcd3235b-kube-api-access-b9khd\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.082656 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.082641    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-data\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.082850 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.082682    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-workload-socket\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.082850 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.082723    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-podinfo\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.082850 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.082744    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90b8fc0b-5847-4799-adb5-141cfcd3235b-istiod-ca-cert\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.183923 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.183826    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-envoy\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.183923 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.183870    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-token\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.183923 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.183909    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9khd\" (UniqueName: \"kubernetes.io/projected/90b8fc0b-5847-4799-adb5-141cfcd3235b-kube-api-access-b9khd\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.184218 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.183991    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-data\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.184218 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.184025    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-workload-socket\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.184218 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.184048    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-podinfo\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.184218 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.184070    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90b8fc0b-5847-4799-adb5-141cfcd3235b-istiod-ca-cert\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.184218 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.184104    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-workload-certs\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.184218 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.184134    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-credential-socket\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.184522 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.184439    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-data\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.184522 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.184489    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-credential-socket\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.184800 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.184669    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-workload-socket\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.184800 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.184734    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-workload-certs\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.184957 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.184843    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90b8fc0b-5847-4799-adb5-141cfcd3235b-istiod-ca-cert\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.186382 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.186358    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-envoy\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.186664 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.186644    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-podinfo\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.192059 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.192033    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-token\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.192179 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.192162    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9khd\" (UniqueName: \"kubernetes.io/projected/90b8fc0b-5847-4799-adb5-141cfcd3235b-kube-api-access-b9khd\") pod \"e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") " pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.248001 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.247969    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:52.391658 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.391629    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"]
Jul 09 12:37:52.392623 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:37:52.392590    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b8fc0b_5847_4799_adb5_141cfcd3235b.slice/crio-9726d40c1c06f36bcffe0b3258efa8b5cd861c80999efeaa7a7661badd5de813 WatchSource:0}: Error finding container 9726d40c1c06f36bcffe0b3258efa8b5cd861c80999efeaa7a7661badd5de813: Status 404 returned error can't find the container with id 9726d40c1c06f36bcffe0b3258efa8b5cd861c80999efeaa7a7661badd5de813
Jul 09 12:37:52.394757 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.394716    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:37:52.394847 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.394794    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:37:52.394847 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.394822    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 12:37:52.996486 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.996453    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6" event={"ID":"90b8fc0b-5847-4799-adb5-141cfcd3235b","Type":"ContainerStarted","Data":"9e83bc483e696e3e2b9bb7488322d675cfb2bdae93c21e57ec4ed57632556983"}
Jul 09 12:37:52.996902 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:52.996507    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6" event={"ID":"90b8fc0b-5847-4799-adb5-141cfcd3235b","Type":"ContainerStarted","Data":"9726d40c1c06f36bcffe0b3258efa8b5cd861c80999efeaa7a7661badd5de813"}
Jul 09 12:37:53.017752 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.017672    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6" podStartSLOduration=2.017652918 podStartE2EDuration="2.017652918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:37:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:37:53.016694501 +0000 UTC m=+2145.694223559" watchObservedRunningTime="2026-07-09 12:37:53.017652918 +0000 UTC m=+2145.695181978"
Jul 09 12:37:53.225812 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.225781    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"]
Jul 09 12:37:53.228454 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.228435    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:53.231382 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.231357    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-9b6bd36b-serving-cert\""
Jul 09 12:37:53.239362 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.239337    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"]
Jul 09 12:37:53.248410 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.248311    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:53.253235 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.253207    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:53.296461 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.296423    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48k8m\" (UniqueName: \"kubernetes.io/projected/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-kube-api-access-48k8m\") pod \"maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2\" (UID: \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:53.296675 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.296488    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-openshift-service-ca\") pod \"maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2\" (UID: \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:53.296675 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.296535    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-maas-api-tls\") pod \"maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2\" (UID: \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:53.397974 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.397934    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-openshift-service-ca\") pod \"maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2\" (UID: \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:53.398186 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.397989    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-maas-api-tls\") pod \"maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2\" (UID: \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:53.398186 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.398072    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48k8m\" (UniqueName: \"kubernetes.io/projected/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-kube-api-access-48k8m\") pod \"maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2\" (UID: \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:53.398186 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:37:53.398159    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-serving-cert: secret "maas-api-e2e-derive-9b6bd36b-serving-cert" not found
Jul 09 12:37:53.398399 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:37:53.398223    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-maas-api-tls podName:e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1 nodeName:}" failed. No retries permitted until 2026-07-09 12:37:53.898207479 +0000 UTC m=+2146.575736516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-maas-api-tls") pod "maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2" (UID: "e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1") : secret "maas-api-e2e-derive-9b6bd36b-serving-cert" not found
Jul 09 12:37:53.398694 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.398670    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-openshift-service-ca\") pod \"maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2\" (UID: \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:53.407353 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.407329    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48k8m\" (UniqueName: \"kubernetes.io/projected/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-kube-api-access-48k8m\") pod \"maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2\" (UID: \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:53.903486 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.903452    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-maas-api-tls\") pod \"maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2\" (UID: \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:53.906032 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:53.906001    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-maas-api-tls\") pod \"maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2\" (UID: \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:54.002382 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:54.002346    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:54.003312 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:54.003290    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:37:54.141097 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:54.141059    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:54.301512 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:37:54.301477    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode35cdeee_e9d3_40e2_9af1_2ad6f8ccd2b1.slice/crio-64682a435c59cc6ccc5eec1811053aa2c79d100e025c6764a822256bea9d839d WatchSource:0}: Error finding container 64682a435c59cc6ccc5eec1811053aa2c79d100e025c6764a822256bea9d839d: Status 404 returned error can't find the container with id 64682a435c59cc6ccc5eec1811053aa2c79d100e025c6764a822256bea9d839d
Jul 09 12:37:54.308046 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:54.308005    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"]
Jul 09 12:37:55.009178 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:55.009085    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2" event={"ID":"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1","Type":"ContainerStarted","Data":"a856f52602801a7a0fd95f3f705d7bcadf9f3571b7cb4a5b8894b0d5c40b0abc"}
Jul 09 12:37:55.009178 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:55.009146    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2" event={"ID":"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1","Type":"ContainerStarted","Data":"64682a435c59cc6ccc5eec1811053aa2c79d100e025c6764a822256bea9d839d"}
Jul 09 12:37:55.028499 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:55.028445    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2" podStartSLOduration=1.717309717 podStartE2EDuration="2.028426963s" totalImagesPullingTime="311.117246ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:37:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:37:55.025431607 +0000 UTC m=+2147.702960665" watchObservedRunningTime="2026-07-09 12:37:55.028426963 +0000 UTC m=+2147.705956021"
Jul 09 12:37:56.015047 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:56.015007    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:56.323330 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:56.323289    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:37:57.659572 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:57.659539    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"]
Jul 09 12:37:57.659995 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:57.659763    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6" podUID="90b8fc0b-5847-4799-adb5-141cfcd3235b" containerName="istio-proxy" containerID="cri-o://9e83bc483e696e3e2b9bb7488322d675cfb2bdae93c21e57ec4ed57632556983" gracePeriod=30
Jul 09 12:37:57.958301 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:57.958194    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"]
Jul 09 12:37:58.033014 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:58.032968    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2" podUID="e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1" containerName="maas-api" containerID="cri-o://a856f52602801a7a0fd95f3f705d7bcadf9f3571b7cb4a5b8894b0d5c40b0abc" gracePeriod=30
Jul 09 12:37:58.311887 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:58.311852    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:58.452797 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:58.452765    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-maas-api-tls\") pod \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\" (UID: \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\") "
Jul 09 12:37:58.452797 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:58.452809    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-openshift-service-ca\") pod \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\" (UID: \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\") "
Jul 09 12:37:58.453047 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:58.452884    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48k8m\" (UniqueName: \"kubernetes.io/projected/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-kube-api-access-48k8m\") pod \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\" (UID: \"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1\") "
Jul 09 12:37:58.453207 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:58.453179    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1" (UID: "e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:37:58.455092 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:58.455068    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-kube-api-access-48k8m" (OuterVolumeSpecName: "kube-api-access-48k8m") pod "e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1" (UID: "e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1"). InnerVolumeSpecName "kube-api-access-48k8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:37:58.455179 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:58.455149    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1" (UID: "e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:37:58.553844 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:58.553739    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48k8m\" (UniqueName: \"kubernetes.io/projected/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-kube-api-access-48k8m\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:58.553844 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:58.553778    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-maas-api-tls\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:58.553844 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:58.553791    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1-openshift-service-ca\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:37:59.044229 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:59.044194    2565 generic.go:356] "Generic (PLEG): container finished" podID="e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1" containerID="a856f52602801a7a0fd95f3f705d7bcadf9f3571b7cb4a5b8894b0d5c40b0abc" exitCode=0
Jul 09 12:37:59.044742 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:59.044241    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"
Jul 09 12:37:59.044742 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:59.044268    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2" event={"ID":"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1","Type":"ContainerDied","Data":"a856f52602801a7a0fd95f3f705d7bcadf9f3571b7cb4a5b8894b0d5c40b0abc"}
Jul 09 12:37:59.044742 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:59.044313    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2" event={"ID":"e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1","Type":"ContainerDied","Data":"64682a435c59cc6ccc5eec1811053aa2c79d100e025c6764a822256bea9d839d"}
Jul 09 12:37:59.044742 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:59.044333    2565 scope.go:117] "RemoveContainer" containerID="a856f52602801a7a0fd95f3f705d7bcadf9f3571b7cb4a5b8894b0d5c40b0abc"
Jul 09 12:37:59.053732 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:59.053691    2565 scope.go:117] "RemoveContainer" containerID="a856f52602801a7a0fd95f3f705d7bcadf9f3571b7cb4a5b8894b0d5c40b0abc"
Jul 09 12:37:59.054910 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:37:59.054887    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a856f52602801a7a0fd95f3f705d7bcadf9f3571b7cb4a5b8894b0d5c40b0abc\": container with ID starting with a856f52602801a7a0fd95f3f705d7bcadf9f3571b7cb4a5b8894b0d5c40b0abc not found: ID does not exist" containerID="a856f52602801a7a0fd95f3f705d7bcadf9f3571b7cb4a5b8894b0d5c40b0abc"
Jul 09 12:37:59.054982 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:59.054926    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a856f52602801a7a0fd95f3f705d7bcadf9f3571b7cb4a5b8894b0d5c40b0abc"} err="failed to get container status \"a856f52602801a7a0fd95f3f705d7bcadf9f3571b7cb4a5b8894b0d5c40b0abc\": rpc error: code = NotFound desc = could not find container \"a856f52602801a7a0fd95f3f705d7bcadf9f3571b7cb4a5b8894b0d5c40b0abc\": container with ID starting with a856f52602801a7a0fd95f3f705d7bcadf9f3571b7cb4a5b8894b0d5c40b0abc not found: ID does not exist"
Jul 09 12:37:59.068077 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:59.068045    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"]
Jul 09 12:37:59.070383 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:59.070352    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9b6bd36b-cf9778878-r8kx2"]
Jul 09 12:37:59.894149 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:37:59.894065    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1" path="/var/lib/kubelet/pods/e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1/volumes"
Jul 09 12:38:04.421502 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.421430    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:38:04.513637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.513533    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-token\") pod \"90b8fc0b-5847-4799-adb5-141cfcd3235b\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") "
Jul 09 12:38:04.513637 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.513590    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-credential-socket\") pod \"90b8fc0b-5847-4799-adb5-141cfcd3235b\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") "
Jul 09 12:38:04.513921 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.513660    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-data\") pod \"90b8fc0b-5847-4799-adb5-141cfcd3235b\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") "
Jul 09 12:38:04.513921 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.513693    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-envoy\") pod \"90b8fc0b-5847-4799-adb5-141cfcd3235b\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") "
Jul 09 12:38:04.513921 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.513747    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9khd\" (UniqueName: \"kubernetes.io/projected/90b8fc0b-5847-4799-adb5-141cfcd3235b-kube-api-access-b9khd\") pod \"90b8fc0b-5847-4799-adb5-141cfcd3235b\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") "
Jul 09 12:38:04.513921 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.513782    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-podinfo\") pod \"90b8fc0b-5847-4799-adb5-141cfcd3235b\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") "
Jul 09 12:38:04.513921 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.513828    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-workload-socket\") pod \"90b8fc0b-5847-4799-adb5-141cfcd3235b\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") "
Jul 09 12:38:04.513921 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.513861    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-workload-certs\") pod \"90b8fc0b-5847-4799-adb5-141cfcd3235b\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") "
Jul 09 12:38:04.513921 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.513896    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90b8fc0b-5847-4799-adb5-141cfcd3235b-istiod-ca-cert\") pod \"90b8fc0b-5847-4799-adb5-141cfcd3235b\" (UID: \"90b8fc0b-5847-4799-adb5-141cfcd3235b\") "
Jul 09 12:38:04.514267 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.513942    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "90b8fc0b-5847-4799-adb5-141cfcd3235b" (UID: "90b8fc0b-5847-4799-adb5-141cfcd3235b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:38:04.514267 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.514106    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "90b8fc0b-5847-4799-adb5-141cfcd3235b" (UID: "90b8fc0b-5847-4799-adb5-141cfcd3235b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:38:04.514267 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.514127    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "90b8fc0b-5847-4799-adb5-141cfcd3235b" (UID: "90b8fc0b-5847-4799-adb5-141cfcd3235b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:38:04.514267 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.514248    2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-credential-socket\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:38:04.514487 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.514288    2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-workload-socket\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:38:04.514487 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.514302    2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-workload-certs\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:38:04.514487 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.514428    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90b8fc0b-5847-4799-adb5-141cfcd3235b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "90b8fc0b-5847-4799-adb5-141cfcd3235b" (UID: "90b8fc0b-5847-4799-adb5-141cfcd3235b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:38:04.514611 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.514497    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-data" (OuterVolumeSpecName: "istio-data") pod "90b8fc0b-5847-4799-adb5-141cfcd3235b" (UID: "90b8fc0b-5847-4799-adb5-141cfcd3235b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:38:04.516262 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.516237    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "90b8fc0b-5847-4799-adb5-141cfcd3235b" (UID: "90b8fc0b-5847-4799-adb5-141cfcd3235b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 09 12:38:04.516365 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.516328    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-token" (OuterVolumeSpecName: "istio-token") pod "90b8fc0b-5847-4799-adb5-141cfcd3235b" (UID: "90b8fc0b-5847-4799-adb5-141cfcd3235b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:38:04.516540 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.516522    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "90b8fc0b-5847-4799-adb5-141cfcd3235b" (UID: "90b8fc0b-5847-4799-adb5-141cfcd3235b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:38:04.516590 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.516540    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90b8fc0b-5847-4799-adb5-141cfcd3235b-kube-api-access-b9khd" (OuterVolumeSpecName: "kube-api-access-b9khd") pod "90b8fc0b-5847-4799-adb5-141cfcd3235b" (UID: "90b8fc0b-5847-4799-adb5-141cfcd3235b"). InnerVolumeSpecName "kube-api-access-b9khd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:38:04.615560 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.615517    2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-token\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:38:04.615560 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.615552    2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-data\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:38:04.615560 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.615564    2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-envoy\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:38:04.615863 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.615576    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9khd\" (UniqueName: \"kubernetes.io/projected/90b8fc0b-5847-4799-adb5-141cfcd3235b-kube-api-access-b9khd\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:38:04.615863 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.615590    2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90b8fc0b-5847-4799-adb5-141cfcd3235b-istio-podinfo\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:38:04.615863 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:04.615602    2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90b8fc0b-5847-4799-adb5-141cfcd3235b-istiod-ca-cert\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:38:05.087262 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:05.087222    2565 generic.go:356] "Generic (PLEG): container finished" podID="90b8fc0b-5847-4799-adb5-141cfcd3235b" containerID="9e83bc483e696e3e2b9bb7488322d675cfb2bdae93c21e57ec4ed57632556983" exitCode=0
Jul 09 12:38:05.087454 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:05.087269    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"
Jul 09 12:38:05.087454 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:05.087302    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6" event={"ID":"90b8fc0b-5847-4799-adb5-141cfcd3235b","Type":"ContainerDied","Data":"9e83bc483e696e3e2b9bb7488322d675cfb2bdae93c21e57ec4ed57632556983"}
Jul 09 12:38:05.087454 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:05.087357    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6" event={"ID":"90b8fc0b-5847-4799-adb5-141cfcd3235b","Type":"ContainerDied","Data":"9726d40c1c06f36bcffe0b3258efa8b5cd861c80999efeaa7a7661badd5de813"}
Jul 09 12:38:05.087454 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:05.087380    2565 scope.go:117] "RemoveContainer" containerID="9e83bc483e696e3e2b9bb7488322d675cfb2bdae93c21e57ec4ed57632556983"
Jul 09 12:38:05.098081 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:05.098060    2565 scope.go:117] "RemoveContainer" containerID="9e83bc483e696e3e2b9bb7488322d675cfb2bdae93c21e57ec4ed57632556983"
Jul 09 12:38:05.098420 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:38:05.098398    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e83bc483e696e3e2b9bb7488322d675cfb2bdae93c21e57ec4ed57632556983\": container with ID starting with 9e83bc483e696e3e2b9bb7488322d675cfb2bdae93c21e57ec4ed57632556983 not found: ID does not exist" containerID="9e83bc483e696e3e2b9bb7488322d675cfb2bdae93c21e57ec4ed57632556983"
Jul 09 12:38:05.098503 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:05.098432    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e83bc483e696e3e2b9bb7488322d675cfb2bdae93c21e57ec4ed57632556983"} err="failed to get container status \"9e83bc483e696e3e2b9bb7488322d675cfb2bdae93c21e57ec4ed57632556983\": rpc error: code = NotFound desc = could not find container \"9e83bc483e696e3e2b9bb7488322d675cfb2bdae93c21e57ec4ed57632556983\": container with ID starting with 9e83bc483e696e3e2b9bb7488322d675cfb2bdae93c21e57ec4ed57632556983 not found: ID does not exist"
Jul 09 12:38:05.120863 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:05.120825    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"]
Jul 09 12:38:05.135266 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:05.135231    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9b6bd36b-openshift-default-69fc55c9bc-xtnn6"]
Jul 09 12:38:05.894091 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:05.894060    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90b8fc0b-5847-4799-adb5-141cfcd3235b" path="/var/lib/kubelet/pods/90b8fc0b-5847-4799-adb5-141cfcd3235b/volumes"
Jul 09 12:38:07.224551 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:38:07.224512    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:39:27.519603 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:27.519558    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:39:36.331095 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:36.331055    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:39:43.955942 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:43.955910    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"]
Jul 09 12:39:43.956400 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:43.956265    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1" containerName="maas-api"
Jul 09 12:39:43.956400 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:43.956276    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1" containerName="maas-api"
Jul 09 12:39:43.956400 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:43.956286    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90b8fc0b-5847-4799-adb5-141cfcd3235b" containerName="istio-proxy"
Jul 09 12:39:43.956400 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:43.956292    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="90b8fc0b-5847-4799-adb5-141cfcd3235b" containerName="istio-proxy"
Jul 09 12:39:43.956400 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:43.956363    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="90b8fc0b-5847-4799-adb5-141cfcd3235b" containerName="istio-proxy"
Jul 09 12:39:43.956400 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:43.956370    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e35cdeee-e9d3-40e2-9af1-2ad6f8ccd2b1" containerName="maas-api"
Jul 09 12:39:43.959327 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:43.959307    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:43.962071 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:43.962049    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2cff3f7c-serving-cert\""
Jul 09 12:39:43.974332 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:43.974303    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"]
Jul 09 12:39:44.056809 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:44.056759    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e680528b-5ce6-4192-be3f-8b9e4259a312-maas-api-tls\") pod \"maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn\" (UID: \"e680528b-5ce6-4192-be3f-8b9e4259a312\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:44.056998 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:44.056912    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e680528b-5ce6-4192-be3f-8b9e4259a312-openshift-service-ca\") pod \"maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn\" (UID: \"e680528b-5ce6-4192-be3f-8b9e4259a312\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:44.057044 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:44.057000    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98b65\" (UniqueName: \"kubernetes.io/projected/e680528b-5ce6-4192-be3f-8b9e4259a312-kube-api-access-98b65\") pod \"maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn\" (UID: \"e680528b-5ce6-4192-be3f-8b9e4259a312\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:44.157973 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:44.157929    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e680528b-5ce6-4192-be3f-8b9e4259a312-openshift-service-ca\") pod \"maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn\" (UID: \"e680528b-5ce6-4192-be3f-8b9e4259a312\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:44.158182 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:44.158015    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98b65\" (UniqueName: \"kubernetes.io/projected/e680528b-5ce6-4192-be3f-8b9e4259a312-kube-api-access-98b65\") pod \"maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn\" (UID: \"e680528b-5ce6-4192-be3f-8b9e4259a312\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:44.158182 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:44.158095    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e680528b-5ce6-4192-be3f-8b9e4259a312-maas-api-tls\") pod \"maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn\" (UID: \"e680528b-5ce6-4192-be3f-8b9e4259a312\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:44.158278 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:39:44.158225    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-serving-cert: secret "maas-api-e2e-mt-2cff3f7c-serving-cert" not found
Jul 09 12:39:44.158313 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:39:44.158296    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e680528b-5ce6-4192-be3f-8b9e4259a312-maas-api-tls podName:e680528b-5ce6-4192-be3f-8b9e4259a312 nodeName:}" failed. No retries permitted until 2026-07-09 12:39:44.65827591 +0000 UTC m=+2257.335804949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e680528b-5ce6-4192-be3f-8b9e4259a312-maas-api-tls") pod "maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn" (UID: "e680528b-5ce6-4192-be3f-8b9e4259a312") : secret "maas-api-e2e-mt-2cff3f7c-serving-cert" not found
Jul 09 12:39:44.158676 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:44.158656    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e680528b-5ce6-4192-be3f-8b9e4259a312-openshift-service-ca\") pod \"maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn\" (UID: \"e680528b-5ce6-4192-be3f-8b9e4259a312\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:44.167670 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:44.167637    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98b65\" (UniqueName: \"kubernetes.io/projected/e680528b-5ce6-4192-be3f-8b9e4259a312-kube-api-access-98b65\") pod \"maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn\" (UID: \"e680528b-5ce6-4192-be3f-8b9e4259a312\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:44.663410 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:44.663370    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e680528b-5ce6-4192-be3f-8b9e4259a312-maas-api-tls\") pod \"maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn\" (UID: \"e680528b-5ce6-4192-be3f-8b9e4259a312\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:44.666003 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:44.665977    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e680528b-5ce6-4192-be3f-8b9e4259a312-maas-api-tls\") pod \"maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn\" (UID: \"e680528b-5ce6-4192-be3f-8b9e4259a312\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:44.869525 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:44.869476    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:45.009117 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:45.009086    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"]
Jul 09 12:39:45.010831 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:39:45.010804    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode680528b_5ce6_4192_be3f_8b9e4259a312.slice/crio-418d6ffbc65bae3050abc488d50799d70c6f837b03b689857a24974899e7f0a6 WatchSource:0}: Error finding container 418d6ffbc65bae3050abc488d50799d70c6f837b03b689857a24974899e7f0a6: Status 404 returned error can't find the container with id 418d6ffbc65bae3050abc488d50799d70c6f837b03b689857a24974899e7f0a6
Jul 09 12:39:45.648077 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:45.648035    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn" event={"ID":"e680528b-5ce6-4192-be3f-8b9e4259a312","Type":"ContainerStarted","Data":"649a50cc137c660795e86f62317c4d8d11482d25cc579e25e209e8332e0f048b"}
Jul 09 12:39:45.648254 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:45.648095    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn" event={"ID":"e680528b-5ce6-4192-be3f-8b9e4259a312","Type":"ContainerStarted","Data":"418d6ffbc65bae3050abc488d50799d70c6f837b03b689857a24974899e7f0a6"}
Jul 09 12:39:45.648254 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:45.648124    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:45.667201 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:45.667145    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn" podStartSLOduration=2.183180892 podStartE2EDuration="2.667128438s" totalImagesPullingTime="483.947546ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:39:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:39:45.663998615 +0000 UTC m=+2258.341527671" watchObservedRunningTime="2026-07-09 12:39:45.667128438 +0000 UTC m=+2258.344657520"
Jul 09 12:39:49.129132 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:49.129094    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:39:51.658616 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:51.658583    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:56.037097 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.037060    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"]
Jul 09 12:39:56.037918 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.037886    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn" podUID="e680528b-5ce6-4192-be3f-8b9e4259a312" containerName="maas-api" containerID="cri-o://649a50cc137c660795e86f62317c4d8d11482d25cc579e25e209e8332e0f048b" gracePeriod=30
Jul 09 12:39:56.294749 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.294665    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:56.379609 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.379552    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e680528b-5ce6-4192-be3f-8b9e4259a312-maas-api-tls\") pod \"e680528b-5ce6-4192-be3f-8b9e4259a312\" (UID: \"e680528b-5ce6-4192-be3f-8b9e4259a312\") "
Jul 09 12:39:56.379846 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.379729    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e680528b-5ce6-4192-be3f-8b9e4259a312-openshift-service-ca\") pod \"e680528b-5ce6-4192-be3f-8b9e4259a312\" (UID: \"e680528b-5ce6-4192-be3f-8b9e4259a312\") "
Jul 09 12:39:56.379846 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.379779    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98b65\" (UniqueName: \"kubernetes.io/projected/e680528b-5ce6-4192-be3f-8b9e4259a312-kube-api-access-98b65\") pod \"e680528b-5ce6-4192-be3f-8b9e4259a312\" (UID: \"e680528b-5ce6-4192-be3f-8b9e4259a312\") "
Jul 09 12:39:56.380041 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.380017    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e680528b-5ce6-4192-be3f-8b9e4259a312-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e680528b-5ce6-4192-be3f-8b9e4259a312" (UID: "e680528b-5ce6-4192-be3f-8b9e4259a312"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:39:56.380123 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.380109    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e680528b-5ce6-4192-be3f-8b9e4259a312-openshift-service-ca\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:39:56.381925 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.381893    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e680528b-5ce6-4192-be3f-8b9e4259a312-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e680528b-5ce6-4192-be3f-8b9e4259a312" (UID: "e680528b-5ce6-4192-be3f-8b9e4259a312"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:39:56.382038 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.381950    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e680528b-5ce6-4192-be3f-8b9e4259a312-kube-api-access-98b65" (OuterVolumeSpecName: "kube-api-access-98b65") pod "e680528b-5ce6-4192-be3f-8b9e4259a312" (UID: "e680528b-5ce6-4192-be3f-8b9e4259a312"). InnerVolumeSpecName "kube-api-access-98b65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:39:56.480775 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.480725    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98b65\" (UniqueName: \"kubernetes.io/projected/e680528b-5ce6-4192-be3f-8b9e4259a312-kube-api-access-98b65\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:39:56.480775 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.480768    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e680528b-5ce6-4192-be3f-8b9e4259a312-maas-api-tls\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:39:56.707745 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.707693    2565 generic.go:356] "Generic (PLEG): container finished" podID="e680528b-5ce6-4192-be3f-8b9e4259a312" containerID="649a50cc137c660795e86f62317c4d8d11482d25cc579e25e209e8332e0f048b" exitCode=0
Jul 09 12:39:56.707946 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.707754    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"
Jul 09 12:39:56.707946 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.707771    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn" event={"ID":"e680528b-5ce6-4192-be3f-8b9e4259a312","Type":"ContainerDied","Data":"649a50cc137c660795e86f62317c4d8d11482d25cc579e25e209e8332e0f048b"}
Jul 09 12:39:56.707946 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.707858    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn" event={"ID":"e680528b-5ce6-4192-be3f-8b9e4259a312","Type":"ContainerDied","Data":"418d6ffbc65bae3050abc488d50799d70c6f837b03b689857a24974899e7f0a6"}
Jul 09 12:39:56.707946 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.707881    2565 scope.go:117] "RemoveContainer" containerID="649a50cc137c660795e86f62317c4d8d11482d25cc579e25e209e8332e0f048b"
Jul 09 12:39:56.717606 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.717580    2565 scope.go:117] "RemoveContainer" containerID="649a50cc137c660795e86f62317c4d8d11482d25cc579e25e209e8332e0f048b"
Jul 09 12:39:56.717955 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:39:56.717936    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"649a50cc137c660795e86f62317c4d8d11482d25cc579e25e209e8332e0f048b\": container with ID starting with 649a50cc137c660795e86f62317c4d8d11482d25cc579e25e209e8332e0f048b not found: ID does not exist" containerID="649a50cc137c660795e86f62317c4d8d11482d25cc579e25e209e8332e0f048b"
Jul 09 12:39:56.718014 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.717964    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"649a50cc137c660795e86f62317c4d8d11482d25cc579e25e209e8332e0f048b"} err="failed to get container status \"649a50cc137c660795e86f62317c4d8d11482d25cc579e25e209e8332e0f048b\": rpc error: code = NotFound desc = could not find container \"649a50cc137c660795e86f62317c4d8d11482d25cc579e25e209e8332e0f048b\": container with ID starting with 649a50cc137c660795e86f62317c4d8d11482d25cc579e25e209e8332e0f048b not found: ID does not exist"
Jul 09 12:39:56.730182 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.730148    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"]
Jul 09 12:39:56.735343 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:56.735316    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2cff3f7c-8d45495cd-pmrhn"]
Jul 09 12:39:57.894453 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:57.894415    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e680528b-5ce6-4192-be3f-8b9e4259a312" path="/var/lib/kubelet/pods/e680528b-5ce6-4192-be3f-8b9e4259a312/volumes"
Jul 09 12:39:59.831017 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:39:59.830979    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:40:25.225781 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.225746    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"]
Jul 09 12:40:25.226195 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.226137    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e680528b-5ce6-4192-be3f-8b9e4259a312" containerName="maas-api"
Jul 09 12:40:25.226195 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.226149    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e680528b-5ce6-4192-be3f-8b9e4259a312" containerName="maas-api"
Jul 09 12:40:25.226267 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.226223    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e680528b-5ce6-4192-be3f-8b9e4259a312" containerName="maas-api"
Jul 09 12:40:25.229232 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.229210    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:40:25.231756 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.231729    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-bc67cdd8-serving-cert\""
Jul 09 12:40:25.239887 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.239862    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"]
Jul 09 12:40:25.255434 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.255402    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wchff\" (UniqueName: \"kubernetes.io/projected/e6828fb5-e65d-4934-9355-2bb2bec5e188-kube-api-access-wchff\") pod \"maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d\" (UID: \"e6828fb5-e65d-4934-9355-2bb2bec5e188\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:40:25.255601 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.255456    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6828fb5-e65d-4934-9355-2bb2bec5e188-openshift-service-ca\") pod \"maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d\" (UID: \"e6828fb5-e65d-4934-9355-2bb2bec5e188\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:40:25.255601 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.255521    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6828fb5-e65d-4934-9355-2bb2bec5e188-maas-api-tls\") pod \"maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d\" (UID: \"e6828fb5-e65d-4934-9355-2bb2bec5e188\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:40:25.357111 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.357065    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6828fb5-e65d-4934-9355-2bb2bec5e188-maas-api-tls\") pod \"maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d\" (UID: \"e6828fb5-e65d-4934-9355-2bb2bec5e188\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:40:25.357317 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.357202    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wchff\" (UniqueName: \"kubernetes.io/projected/e6828fb5-e65d-4934-9355-2bb2bec5e188-kube-api-access-wchff\") pod \"maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d\" (UID: \"e6828fb5-e65d-4934-9355-2bb2bec5e188\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:40:25.357317 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:40:25.357224    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-serving-cert: secret "maas-api-e2e-mt-bc67cdd8-serving-cert" not found
Jul 09 12:40:25.357317 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.357245    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6828fb5-e65d-4934-9355-2bb2bec5e188-openshift-service-ca\") pod \"maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d\" (UID: \"e6828fb5-e65d-4934-9355-2bb2bec5e188\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:40:25.357317 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:40:25.357286    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6828fb5-e65d-4934-9355-2bb2bec5e188-maas-api-tls podName:e6828fb5-e65d-4934-9355-2bb2bec5e188 nodeName:}" failed. No retries permitted until 2026-07-09 12:40:25.857270098 +0000 UTC m=+2298.534799136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e6828fb5-e65d-4934-9355-2bb2bec5e188-maas-api-tls") pod "maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d" (UID: "e6828fb5-e65d-4934-9355-2bb2bec5e188") : secret "maas-api-e2e-mt-bc67cdd8-serving-cert" not found
Jul 09 12:40:25.357961 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.357938    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6828fb5-e65d-4934-9355-2bb2bec5e188-openshift-service-ca\") pod \"maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d\" (UID: \"e6828fb5-e65d-4934-9355-2bb2bec5e188\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:40:25.369330 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.369295    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wchff\" (UniqueName: \"kubernetes.io/projected/e6828fb5-e65d-4934-9355-2bb2bec5e188-kube-api-access-wchff\") pod \"maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d\" (UID: \"e6828fb5-e65d-4934-9355-2bb2bec5e188\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:40:25.862081 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.862041    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6828fb5-e65d-4934-9355-2bb2bec5e188-maas-api-tls\") pod \"maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d\" (UID: \"e6828fb5-e65d-4934-9355-2bb2bec5e188\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:40:25.864758 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:25.864733    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6828fb5-e65d-4934-9355-2bb2bec5e188-maas-api-tls\") pod \"maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d\" (UID: \"e6828fb5-e65d-4934-9355-2bb2bec5e188\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:40:26.141970 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:26.141862    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:40:26.285633 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:26.285602    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"]
Jul 09 12:40:26.287631 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:40:26.287604    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6828fb5_e65d_4934_9355_2bb2bec5e188.slice/crio-7bf5f75a845a972a437b0aa7121f47a915ad1c0424672755d443f03be6777e01 WatchSource:0}: Error finding container 7bf5f75a845a972a437b0aa7121f47a915ad1c0424672755d443f03be6777e01: Status 404 returned error can't find the container with id 7bf5f75a845a972a437b0aa7121f47a915ad1c0424672755d443f03be6777e01
Jul 09 12:40:26.875136 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:26.875100    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d" event={"ID":"e6828fb5-e65d-4934-9355-2bb2bec5e188","Type":"ContainerStarted","Data":"1446634a3d07157c9e4ec1eacacfda9411178601ddab2ff6508727785898a4c1"}
Jul 09 12:40:26.875306 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:26.875181    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d" event={"ID":"e6828fb5-e65d-4934-9355-2bb2bec5e188","Type":"ContainerStarted","Data":"7bf5f75a845a972a437b0aa7121f47a915ad1c0424672755d443f03be6777e01"}
Jul 09 12:40:26.875306 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:26.875221    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:40:26.894118 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:26.894065    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d" podStartSLOduration=1.503413461 podStartE2EDuration="1.894050111s" totalImagesPullingTime="390.63665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:40:26.891981568 +0000 UTC m=+2299.569510625" watchObservedRunningTime="2026-07-09 12:40:26.894050111 +0000 UTC m=+2299.571579167"
Jul 09 12:40:30.920063 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:30.920026    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:40:32.886853 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:32.886822    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:40:43.430172 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.430134    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"]
Jul 09 12:40:43.433752 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.433727    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.437587 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.437560    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-bc67cdd8\"/\"default-dockercfg-j9qft\""
Jul 09 12:40:43.437835 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.437795    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-bc67cdd8\"/\"e2e-lifecycle-model-bc67cdd8-kserve-self-signed-certs\""
Jul 09 12:40:43.437975 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.437560    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-bc67cdd8\"/\"openshift-service-ca.crt\""
Jul 09 12:40:43.437975 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.437592    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-bc67cdd8\"/\"kube-root-ca.crt\""
Jul 09 12:40:43.444737 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.444656    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"]
Jul 09 12:40:43.529796 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.529750    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-home\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.529796 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.529796    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-tls-certs\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.530032 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.529916    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-dshm\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.530032 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.529969    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-model-cache\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.530032 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.530002    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-tmp-dir\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.530032 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.530027    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxpj8\" (UniqueName: \"kubernetes.io/projected/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-kube-api-access-hxpj8\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.630996 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.630957    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-dshm\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.631191 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.631013    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-model-cache\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.631191 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.631035    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-tmp-dir\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.631191 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.631057    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxpj8\" (UniqueName: \"kubernetes.io/projected/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-kube-api-access-hxpj8\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.631191 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.631117    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-home\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.631191 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.631137    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-tls-certs\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.631516 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.631441    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-model-cache\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.631516 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.631498    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-tmp-dir\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.631606 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.631555    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-home\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.633430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.633398    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-dshm\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.633785 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.633766    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-tls-certs\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.639438 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.639408    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxpj8\" (UniqueName: \"kubernetes.io/projected/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-kube-api-access-hxpj8\") pod \"e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") " pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.744839 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.744715    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:43.886808 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.886774    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"]
Jul 09 12:40:43.888311 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:40:43.888285    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef3b912b_7233_4d1f_89eb_7445c10f6d2e.slice/crio-40502375697b77882011edd907f32e566b3ada23234f16174b87e177851faf6c WatchSource:0}: Error finding container 40502375697b77882011edd907f32e566b3ada23234f16174b87e177851faf6c: Status 404 returned error can't find the container with id 40502375697b77882011edd907f32e566b3ada23234f16174b87e177851faf6c
Jul 09 12:40:43.975391 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.975351    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp" event={"ID":"ef3b912b-7233-4d1f-89eb-7445c10f6d2e","Type":"ContainerStarted","Data":"4af626f4fd297a090b777abce8c60d9572e66590ba0173ccdb100650778505a2"}
Jul 09 12:40:43.975553 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:43.975401    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp" event={"ID":"ef3b912b-7233-4d1f-89eb-7445c10f6d2e","Type":"ContainerStarted","Data":"40502375697b77882011edd907f32e566b3ada23234f16174b87e177851faf6c"}
Jul 09 12:40:44.997374 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:44.997307    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp" podStartSLOduration=1.997290767 podStartE2EDuration="1.997290767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:40:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:40:44.996140228 +0000 UTC m=+2317.673669286" watchObservedRunningTime="2026-07-09 12:40:44.997290767 +0000 UTC m=+2317.674819825"
Jul 09 12:40:47.115887 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:47.115845    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:40:53.744995 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:53.744945    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:53.745462 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:53.745015    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:53.757960 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:53.757924    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:40:54.040613 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:40:54.040535    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:41:11.721432 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:11.721334    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:41:17.721976 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:17.721942    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"]
Jul 09 12:41:17.722429 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:17.722155    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d" podUID="e6828fb5-e65d-4934-9355-2bb2bec5e188" containerName="maas-api" containerID="cri-o://1446634a3d07157c9e4ec1eacacfda9411178601ddab2ff6508727785898a4c1" gracePeriod=30
Jul 09 12:41:17.882104 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:17.882051    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d" podUID="e6828fb5-e65d-4934-9355-2bb2bec5e188" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.51:8443/health\": dial tcp 10.133.0.51:8443: connect: connection refused"
Jul 09 12:41:17.980806 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:17.980719    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:41:18.060094 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.060053    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wchff\" (UniqueName: \"kubernetes.io/projected/e6828fb5-e65d-4934-9355-2bb2bec5e188-kube-api-access-wchff\") pod \"e6828fb5-e65d-4934-9355-2bb2bec5e188\" (UID: \"e6828fb5-e65d-4934-9355-2bb2bec5e188\") "
Jul 09 12:41:18.060270 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.060113    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6828fb5-e65d-4934-9355-2bb2bec5e188-openshift-service-ca\") pod \"e6828fb5-e65d-4934-9355-2bb2bec5e188\" (UID: \"e6828fb5-e65d-4934-9355-2bb2bec5e188\") "
Jul 09 12:41:18.060270 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.060223    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6828fb5-e65d-4934-9355-2bb2bec5e188-maas-api-tls\") pod \"e6828fb5-e65d-4934-9355-2bb2bec5e188\" (UID: \"e6828fb5-e65d-4934-9355-2bb2bec5e188\") "
Jul 09 12:41:18.060463 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.060444    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6828fb5-e65d-4934-9355-2bb2bec5e188-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e6828fb5-e65d-4934-9355-2bb2bec5e188" (UID: "e6828fb5-e65d-4934-9355-2bb2bec5e188"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:41:18.060644 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.060628    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6828fb5-e65d-4934-9355-2bb2bec5e188-openshift-service-ca\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:41:18.062347 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.062325    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6828fb5-e65d-4934-9355-2bb2bec5e188-kube-api-access-wchff" (OuterVolumeSpecName: "kube-api-access-wchff") pod "e6828fb5-e65d-4934-9355-2bb2bec5e188" (UID: "e6828fb5-e65d-4934-9355-2bb2bec5e188"). InnerVolumeSpecName "kube-api-access-wchff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:41:18.062507 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.062488    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6828fb5-e65d-4934-9355-2bb2bec5e188-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e6828fb5-e65d-4934-9355-2bb2bec5e188" (UID: "e6828fb5-e65d-4934-9355-2bb2bec5e188"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:41:18.161293 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.161258    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6828fb5-e65d-4934-9355-2bb2bec5e188-maas-api-tls\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:41:18.161293 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.161292    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wchff\" (UniqueName: \"kubernetes.io/projected/e6828fb5-e65d-4934-9355-2bb2bec5e188-kube-api-access-wchff\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:41:18.162275 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.162251    2565 generic.go:356] "Generic (PLEG): container finished" podID="e6828fb5-e65d-4934-9355-2bb2bec5e188" containerID="1446634a3d07157c9e4ec1eacacfda9411178601ddab2ff6508727785898a4c1" exitCode=0
Jul 09 12:41:18.162331 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.162303    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"
Jul 09 12:41:18.162374 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.162343    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d" event={"ID":"e6828fb5-e65d-4934-9355-2bb2bec5e188","Type":"ContainerDied","Data":"1446634a3d07157c9e4ec1eacacfda9411178601ddab2ff6508727785898a4c1"}
Jul 09 12:41:18.162416 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.162403    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d" event={"ID":"e6828fb5-e65d-4934-9355-2bb2bec5e188","Type":"ContainerDied","Data":"7bf5f75a845a972a437b0aa7121f47a915ad1c0424672755d443f03be6777e01"}
Jul 09 12:41:18.162452 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.162436    2565 scope.go:117] "RemoveContainer" containerID="1446634a3d07157c9e4ec1eacacfda9411178601ddab2ff6508727785898a4c1"
Jul 09 12:41:18.172724 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.172684    2565 scope.go:117] "RemoveContainer" containerID="1446634a3d07157c9e4ec1eacacfda9411178601ddab2ff6508727785898a4c1"
Jul 09 12:41:18.173022 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:41:18.173000    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1446634a3d07157c9e4ec1eacacfda9411178601ddab2ff6508727785898a4c1\": container with ID starting with 1446634a3d07157c9e4ec1eacacfda9411178601ddab2ff6508727785898a4c1 not found: ID does not exist" containerID="1446634a3d07157c9e4ec1eacacfda9411178601ddab2ff6508727785898a4c1"
Jul 09 12:41:18.173085 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.173031    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1446634a3d07157c9e4ec1eacacfda9411178601ddab2ff6508727785898a4c1"} err="failed to get container status \"1446634a3d07157c9e4ec1eacacfda9411178601ddab2ff6508727785898a4c1\": rpc error: code = NotFound desc = could not find container \"1446634a3d07157c9e4ec1eacacfda9411178601ddab2ff6508727785898a4c1\": container with ID starting with 1446634a3d07157c9e4ec1eacacfda9411178601ddab2ff6508727785898a4c1 not found: ID does not exist"
Jul 09 12:41:18.185006 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.184969    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"]
Jul 09 12:41:18.189571 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:18.189542    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bc67cdd8-5789bc954c-lnq6d"]
Jul 09 12:41:19.897469 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:19.897429    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6828fb5-e65d-4934-9355-2bb2bec5e188" path="/var/lib/kubelet/pods/e6828fb5-e65d-4934-9355-2bb2bec5e188/volumes"
Jul 09 12:41:22.220304 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:22.220266    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:41:28.376732 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:28.376658    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"]
Jul 09 12:41:32.121468 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:32.121421    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:41:43.397731 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.397651    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp" podUID="ef3b912b-7233-4d1f-89eb-7445c10f6d2e" containerName="main" containerID="cri-o://4af626f4fd297a090b777abce8c60d9572e66590ba0173ccdb100650778505a2" gracePeriod=45
Jul 09 12:41:43.662339 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.662276    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:41:43.701885 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.701851    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-tls-certs\") pod \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") "
Jul 09 12:41:43.702081 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.701905    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxpj8\" (UniqueName: \"kubernetes.io/projected/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-kube-api-access-hxpj8\") pod \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") "
Jul 09 12:41:43.702081 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.701955    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-dshm\") pod \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") "
Jul 09 12:41:43.702081 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.701991    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-tmp-dir\") pod \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") "
Jul 09 12:41:43.702261 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.702142    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-model-cache\") pod \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") "
Jul 09 12:41:43.702324 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.702277    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ef3b912b-7233-4d1f-89eb-7445c10f6d2e" (UID: "ef3b912b-7233-4d1f-89eb-7445c10f6d2e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:41:43.702535 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.702496    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-model-cache" (OuterVolumeSpecName: "model-cache") pod "ef3b912b-7233-4d1f-89eb-7445c10f6d2e" (UID: "ef3b912b-7233-4d1f-89eb-7445c10f6d2e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:41:43.702672 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.702567    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-tmp-dir\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:41:43.704538 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.704513    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ef3b912b-7233-4d1f-89eb-7445c10f6d2e" (UID: "ef3b912b-7233-4d1f-89eb-7445c10f6d2e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:41:43.704741 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.704723    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-dshm" (OuterVolumeSpecName: "dshm") pod "ef3b912b-7233-4d1f-89eb-7445c10f6d2e" (UID: "ef3b912b-7233-4d1f-89eb-7445c10f6d2e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:41:43.704847 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.704824    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-kube-api-access-hxpj8" (OuterVolumeSpecName: "kube-api-access-hxpj8") pod "ef3b912b-7233-4d1f-89eb-7445c10f6d2e" (UID: "ef3b912b-7233-4d1f-89eb-7445c10f6d2e"). InnerVolumeSpecName "kube-api-access-hxpj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:41:43.803276 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.803244    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-home\") pod \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\" (UID: \"ef3b912b-7233-4d1f-89eb-7445c10f6d2e\") "
Jul 09 12:41:43.803487 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.803473    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-tls-certs\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:41:43.803531 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.803491    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxpj8\" (UniqueName: \"kubernetes.io/projected/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-kube-api-access-hxpj8\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:41:43.803531 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.803500    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-dshm\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:41:43.803531 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.803508    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-model-cache\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:41:43.803627 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.803527    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-home" (OuterVolumeSpecName: "home") pod "ef3b912b-7233-4d1f-89eb-7445c10f6d2e" (UID: "ef3b912b-7233-4d1f-89eb-7445c10f6d2e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:41:43.904963 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:43.904939    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef3b912b-7233-4d1f-89eb-7445c10f6d2e-home\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:41:44.306576 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:44.306541    2565 generic.go:356] "Generic (PLEG): container finished" podID="ef3b912b-7233-4d1f-89eb-7445c10f6d2e" containerID="4af626f4fd297a090b777abce8c60d9572e66590ba0173ccdb100650778505a2" exitCode=0
Jul 09 12:41:44.306778 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:44.306586    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"
Jul 09 12:41:44.306778 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:44.306610    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp" event={"ID":"ef3b912b-7233-4d1f-89eb-7445c10f6d2e","Type":"ContainerDied","Data":"4af626f4fd297a090b777abce8c60d9572e66590ba0173ccdb100650778505a2"}
Jul 09 12:41:44.306778 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:44.306650    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp" event={"ID":"ef3b912b-7233-4d1f-89eb-7445c10f6d2e","Type":"ContainerDied","Data":"40502375697b77882011edd907f32e566b3ada23234f16174b87e177851faf6c"}
Jul 09 12:41:44.306778 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:44.306670    2565 scope.go:117] "RemoveContainer" containerID="4af626f4fd297a090b777abce8c60d9572e66590ba0173ccdb100650778505a2"
Jul 09 12:41:44.316155 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:44.316135    2565 scope.go:117] "RemoveContainer" containerID="4af626f4fd297a090b777abce8c60d9572e66590ba0173ccdb100650778505a2"
Jul 09 12:41:44.316499 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:41:44.316476    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4af626f4fd297a090b777abce8c60d9572e66590ba0173ccdb100650778505a2\": container with ID starting with 4af626f4fd297a090b777abce8c60d9572e66590ba0173ccdb100650778505a2 not found: ID does not exist" containerID="4af626f4fd297a090b777abce8c60d9572e66590ba0173ccdb100650778505a2"
Jul 09 12:41:44.316545 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:44.316500    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4af626f4fd297a090b777abce8c60d9572e66590ba0173ccdb100650778505a2"} err="failed to get container status \"4af626f4fd297a090b777abce8c60d9572e66590ba0173ccdb100650778505a2\": rpc error: code = NotFound desc = could not find container \"4af626f4fd297a090b777abce8c60d9572e66590ba0173ccdb100650778505a2\": container with ID starting with 4af626f4fd297a090b777abce8c60d9572e66590ba0173ccdb100650778505a2 not found: ID does not exist"
Jul 09 12:41:44.327761 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:44.327726    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"]
Jul 09 12:41:44.331873 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:44.331841    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-bc67cdd8/e2e-lifecycle-model-bc67cdd8-kserve-7894cc7ff9-zsxgp"]
Jul 09 12:41:45.894897 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:41:45.894856    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef3b912b-7233-4d1f-89eb-7445c10f6d2e" path="/var/lib/kubelet/pods/ef3b912b-7233-4d1f-89eb-7445c10f6d2e/volumes"
Jul 09 12:42:08.030886 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:42:08.030857    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:42:08.043115 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:42:08.043087    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:42:08.541443 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:42:08.541407    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:42:17.936716 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:42:17.936676    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:42:31.232416 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:42:31.232373    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:43:12.149733 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.149677    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"]
Jul 09 12:43:12.152237 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.150062    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef3b912b-7233-4d1f-89eb-7445c10f6d2e" containerName="main"
Jul 09 12:43:12.152237 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.150074    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef3b912b-7233-4d1f-89eb-7445c10f6d2e" containerName="main"
Jul 09 12:43:12.152237 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.150099    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6828fb5-e65d-4934-9355-2bb2bec5e188" containerName="maas-api"
Jul 09 12:43:12.152237 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.150106    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6828fb5-e65d-4934-9355-2bb2bec5e188" containerName="maas-api"
Jul 09 12:43:12.152237 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.150163    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef3b912b-7233-4d1f-89eb-7445c10f6d2e" containerName="main"
Jul 09 12:43:12.152237 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.150172    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6828fb5-e65d-4934-9355-2bb2bec5e188" containerName="maas-api"
Jul 09 12:43:12.153266 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.153244    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:43:12.155915 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.155886    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-4861fa-serving-cert\""
Jul 09 12:43:12.166158 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.166129    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"]
Jul 09 12:43:12.239589 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.239542    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9v9w\" (UniqueName: \"kubernetes.io/projected/cde17633-1a9e-4744-b564-ebe0ef11e363-kube-api-access-l9v9w\") pod \"maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w\" (UID: \"cde17633-1a9e-4744-b564-ebe0ef11e363\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:43:12.239817 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.239610    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde17633-1a9e-4744-b564-ebe0ef11e363-maas-api-tls\") pod \"maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w\" (UID: \"cde17633-1a9e-4744-b564-ebe0ef11e363\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:43:12.239817 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.239635    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde17633-1a9e-4744-b564-ebe0ef11e363-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w\" (UID: \"cde17633-1a9e-4744-b564-ebe0ef11e363\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:43:12.340537 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.340496    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde17633-1a9e-4744-b564-ebe0ef11e363-maas-api-tls\") pod \"maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w\" (UID: \"cde17633-1a9e-4744-b564-ebe0ef11e363\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:43:12.340537 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.340542    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde17633-1a9e-4744-b564-ebe0ef11e363-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w\" (UID: \"cde17633-1a9e-4744-b564-ebe0ef11e363\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:43:12.340871 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.340642    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9v9w\" (UniqueName: \"kubernetes.io/projected/cde17633-1a9e-4744-b564-ebe0ef11e363-kube-api-access-l9v9w\") pod \"maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w\" (UID: \"cde17633-1a9e-4744-b564-ebe0ef11e363\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:43:12.340871 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:43:12.340683    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-serving-cert: secret "maas-api-e2e-inf-a-4861fa-serving-cert" not found
Jul 09 12:43:12.340871 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:43:12.340807    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cde17633-1a9e-4744-b564-ebe0ef11e363-maas-api-tls podName:cde17633-1a9e-4744-b564-ebe0ef11e363 nodeName:}" failed. No retries permitted until 2026-07-09 12:43:12.840782559 +0000 UTC m=+2465.518311596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cde17633-1a9e-4744-b564-ebe0ef11e363-maas-api-tls") pod "maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w" (UID: "cde17633-1a9e-4744-b564-ebe0ef11e363") : secret "maas-api-e2e-inf-a-4861fa-serving-cert" not found
Jul 09 12:43:12.341390 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.341365    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde17633-1a9e-4744-b564-ebe0ef11e363-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w\" (UID: \"cde17633-1a9e-4744-b564-ebe0ef11e363\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:43:12.350353 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.350326    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9v9w\" (UniqueName: \"kubernetes.io/projected/cde17633-1a9e-4744-b564-ebe0ef11e363-kube-api-access-l9v9w\") pod \"maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w\" (UID: \"cde17633-1a9e-4744-b564-ebe0ef11e363\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:43:12.844944 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.844906    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde17633-1a9e-4744-b564-ebe0ef11e363-maas-api-tls\") pod \"maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w\" (UID: \"cde17633-1a9e-4744-b564-ebe0ef11e363\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:43:12.847511 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:12.847492    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde17633-1a9e-4744-b564-ebe0ef11e363-maas-api-tls\") pod \"maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w\" (UID: \"cde17633-1a9e-4744-b564-ebe0ef11e363\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:43:13.064946 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:13.064908    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:43:13.205850 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:13.205820    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"]
Jul 09 12:43:13.207642 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:43:13.207606    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcde17633_1a9e_4744_b564_ebe0ef11e363.slice/crio-3c6e8e88e51d05c4d52dad70e862af7cdbbf9569bf85586d98bf241f00975bf1 WatchSource:0}: Error finding container 3c6e8e88e51d05c4d52dad70e862af7cdbbf9569bf85586d98bf241f00975bf1: Status 404 returned error can't find the container with id 3c6e8e88e51d05c4d52dad70e862af7cdbbf9569bf85586d98bf241f00975bf1
Jul 09 12:43:13.209147 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:13.209121    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 12:43:13.789783 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:13.789746    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w" event={"ID":"cde17633-1a9e-4744-b564-ebe0ef11e363","Type":"ContainerStarted","Data":"e5cdbd55a665f1c89b16bf0232e01cda3f2b57a8beff7e2d579001f5ad861785"}
Jul 09 12:43:13.789963 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:13.789804    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w" event={"ID":"cde17633-1a9e-4744-b564-ebe0ef11e363","Type":"ContainerStarted","Data":"3c6e8e88e51d05c4d52dad70e862af7cdbbf9569bf85586d98bf241f00975bf1"}
Jul 09 12:43:13.789963 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:13.789876    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:43:13.807966 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:13.807902    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w" podStartSLOduration=1.311297875 podStartE2EDuration="1.807885043s" totalImagesPullingTime="496.587168ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:43:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:43:13.805636452 +0000 UTC m=+2466.483165513" watchObservedRunningTime="2026-07-09 12:43:13.807885043 +0000 UTC m=+2466.485414101"
Jul 09 12:43:19.802275 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:19.802242    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:43:20.161416 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.161374    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"]
Jul 09 12:43:20.165745 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.165719    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:43:20.168394 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.168360    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a2f2c0-serving-cert\""
Jul 09 12:43:20.174378 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.174061    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"]
Jul 09 12:43:20.216911 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.216874    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6be2862-46db-46cf-8b45-bd371f103a40-maas-api-tls\") pod \"maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp\" (UID: \"e6be2862-46db-46cf-8b45-bd371f103a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:43:20.216911 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.216913    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6be2862-46db-46cf-8b45-bd371f103a40-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp\" (UID: \"e6be2862-46db-46cf-8b45-bd371f103a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:43:20.217125 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.217045    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsh9k\" (UniqueName: \"kubernetes.io/projected/e6be2862-46db-46cf-8b45-bd371f103a40-kube-api-access-wsh9k\") pod \"maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp\" (UID: \"e6be2862-46db-46cf-8b45-bd371f103a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:43:20.318682 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.318643    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsh9k\" (UniqueName: \"kubernetes.io/projected/e6be2862-46db-46cf-8b45-bd371f103a40-kube-api-access-wsh9k\") pod \"maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp\" (UID: \"e6be2862-46db-46cf-8b45-bd371f103a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:43:20.318912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.318763    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6be2862-46db-46cf-8b45-bd371f103a40-maas-api-tls\") pod \"maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp\" (UID: \"e6be2862-46db-46cf-8b45-bd371f103a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:43:20.318912 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.318856    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6be2862-46db-46cf-8b45-bd371f103a40-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp\" (UID: \"e6be2862-46db-46cf-8b45-bd371f103a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:43:20.319097 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:43:20.318878    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-serving-cert: secret "maas-api-e2e-inf-b-a2f2c0-serving-cert" not found
Jul 09 12:43:20.319195 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:43:20.319180    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6be2862-46db-46cf-8b45-bd371f103a40-maas-api-tls podName:e6be2862-46db-46cf-8b45-bd371f103a40 nodeName:}" failed. No retries permitted until 2026-07-09 12:43:20.819155751 +0000 UTC m=+2473.496684791 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e6be2862-46db-46cf-8b45-bd371f103a40-maas-api-tls") pod "maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp" (UID: "e6be2862-46db-46cf-8b45-bd371f103a40") : secret "maas-api-e2e-inf-b-a2f2c0-serving-cert" not found
Jul 09 12:43:20.319796 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.319771    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6be2862-46db-46cf-8b45-bd371f103a40-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp\" (UID: \"e6be2862-46db-46cf-8b45-bd371f103a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:43:20.321806 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.321772    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:43:20.329777 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.329749    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsh9k\" (UniqueName: \"kubernetes.io/projected/e6be2862-46db-46cf-8b45-bd371f103a40-kube-api-access-wsh9k\") pod \"maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp\" (UID: \"e6be2862-46db-46cf-8b45-bd371f103a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:43:20.824511 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.824465    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6be2862-46db-46cf-8b45-bd371f103a40-maas-api-tls\") pod \"maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp\" (UID: \"e6be2862-46db-46cf-8b45-bd371f103a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:43:20.827243 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:20.827215    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6be2862-46db-46cf-8b45-bd371f103a40-maas-api-tls\") pod \"maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp\" (UID: \"e6be2862-46db-46cf-8b45-bd371f103a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:43:21.078865 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:21.078764    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:43:21.227193 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:21.227153    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"]
Jul 09 12:43:21.228162 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:43:21.228138    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6be2862_46db_46cf_8b45_bd371f103a40.slice/crio-c75830a84c0933ddfc9d86b3a1efdef00176ed3eaa012d35a07f474d26c76c63 WatchSource:0}: Error finding container c75830a84c0933ddfc9d86b3a1efdef00176ed3eaa012d35a07f474d26c76c63: Status 404 returned error can't find the container with id c75830a84c0933ddfc9d86b3a1efdef00176ed3eaa012d35a07f474d26c76c63
Jul 09 12:43:21.848562 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:21.848521    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp" event={"ID":"e6be2862-46db-46cf-8b45-bd371f103a40","Type":"ContainerStarted","Data":"914eb88a4678adcc4c03a3673536efa350d90f0698d5918c65b0f3c49a876b81"}
Jul 09 12:43:21.849029 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:21.848665    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:43:21.849029 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:21.848688    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp" event={"ID":"e6be2862-46db-46cf-8b45-bd371f103a40","Type":"ContainerStarted","Data":"c75830a84c0933ddfc9d86b3a1efdef00176ed3eaa012d35a07f474d26c76c63"}
Jul 09 12:43:21.868350 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:21.868296    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp" podStartSLOduration=1.38615904 podStartE2EDuration="1.868278229s" totalImagesPullingTime="482.119189ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:43:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:43:21.867335177 +0000 UTC m=+2474.544864267" watchObservedRunningTime="2026-07-09 12:43:21.868278229 +0000 UTC m=+2474.545807286"
Jul 09 12:43:27.859540 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:27.859506    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:43:35.317960 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:35.317911    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:43:44.331344 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:44.331298    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:43:58.627670 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.627636    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"]
Jul 09 12:43:58.631452 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.631433    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.635668 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.635642    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a2f2c0\"/\"test-model-a2f2c0-kserve-self-signed-certs\""
Jul 09 12:43:58.635824 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.635642    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a2f2c0\"/\"default-dockercfg-d6lks\""
Jul 09 12:43:58.635824 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.635645    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a2f2c0\"/\"openshift-service-ca.crt\""
Jul 09 12:43:58.635824 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.635675    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a2f2c0\"/\"kube-root-ca.crt\""
Jul 09 12:43:58.639763 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.639742    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"]
Jul 09 12:43:58.686930 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.686898    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-dshm\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.687111 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.686948    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41ad6072-5084-4810-9ac1-50503305727d-tls-certs\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.687111 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.687020    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-home\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.687111 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.687067    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d65pj\" (UniqueName: \"kubernetes.io/projected/41ad6072-5084-4810-9ac1-50503305727d-kube-api-access-d65pj\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.687111 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.687092    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-tmp-dir\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.687266 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.687132    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-model-cache\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.788201 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.788106    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-home\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.788201 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.788148    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d65pj\" (UniqueName: \"kubernetes.io/projected/41ad6072-5084-4810-9ac1-50503305727d-kube-api-access-d65pj\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.788201 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.788180    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-tmp-dir\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.788493 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.788234    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-model-cache\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.788493 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.788329    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-dshm\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.788493 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.788367    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41ad6072-5084-4810-9ac1-50503305727d-tls-certs\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.788673 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.788621    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-home\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.788673 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.788658    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-tmp-dir\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.788825 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.788796    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-model-cache\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.790859 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.790820    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-dshm\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.791127 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.791107    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41ad6072-5084-4810-9ac1-50503305727d-tls-certs\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.796594 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.796570    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d65pj\" (UniqueName: \"kubernetes.io/projected/41ad6072-5084-4810-9ac1-50503305727d-kube-api-access-d65pj\") pod \"test-model-a2f2c0-kserve-58bdf9b445-24x79\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") " pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:58.941584 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:58.941485    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:43:59.023389 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:59.023356    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:43:59.088427 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:43:59.088402    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"]
Jul 09 12:43:59.089936 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:43:59.089898    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41ad6072_5084_4810_9ac1_50503305727d.slice/crio-12c464864ade4da5a65b0df1ab2317881df512b980f6f95f692f75b04125f93f WatchSource:0}: Error finding container 12c464864ade4da5a65b0df1ab2317881df512b980f6f95f692f75b04125f93f: Status 404 returned error can't find the container with id 12c464864ade4da5a65b0df1ab2317881df512b980f6f95f692f75b04125f93f
Jul 09 12:44:00.061016 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:00.060979    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79" event={"ID":"41ad6072-5084-4810-9ac1-50503305727d","Type":"ContainerStarted","Data":"9ff2b6b030f1b2ed40dd7181761228e6f3c6528dfc0b887b71be07cf34e71264"}
Jul 09 12:44:00.061016 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:00.061020    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79" event={"ID":"41ad6072-5084-4810-9ac1-50503305727d","Type":"ContainerStarted","Data":"12c464864ade4da5a65b0df1ab2317881df512b980f6f95f692f75b04125f93f"}
Jul 09 12:44:00.080600 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:00.080543    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79" podStartSLOduration=2.080528964 podStartE2EDuration="2.080528964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:43:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:44:00.077968788 +0000 UTC m=+2512.755497877" watchObservedRunningTime="2026-07-09 12:44:00.080528964 +0000 UTC m=+2512.758058021"
Jul 09 12:44:08.941955 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:08.941914    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:44:08.941955 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:08.941965    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:44:08.954510 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:08.954485    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:44:09.122817 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:09.122787    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:44:16.618261 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:16.618211    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:44:44.247669 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:44.247621    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"]
Jul 09 12:44:45.189969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:45.189932    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"]
Jul 09 12:44:45.190219 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:45.190193    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w" podUID="cde17633-1a9e-4744-b564-ebe0ef11e363" containerName="maas-api" containerID="cri-o://e5cdbd55a665f1c89b16bf0232e01cda3f2b57a8beff7e2d579001f5ad861785" gracePeriod=30
Jul 09 12:44:45.321189 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:45.321142    2565 generic.go:356] "Generic (PLEG): container finished" podID="cde17633-1a9e-4744-b564-ebe0ef11e363" containerID="e5cdbd55a665f1c89b16bf0232e01cda3f2b57a8beff7e2d579001f5ad861785" exitCode=0
Jul 09 12:44:45.321189 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:45.321188    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w" event={"ID":"cde17633-1a9e-4744-b564-ebe0ef11e363","Type":"ContainerDied","Data":"e5cdbd55a665f1c89b16bf0232e01cda3f2b57a8beff7e2d579001f5ad861785"}
Jul 09 12:44:45.447847 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:45.447778    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:44:45.537906 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:45.537873    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde17633-1a9e-4744-b564-ebe0ef11e363-openshift-service-ca\") pod \"cde17633-1a9e-4744-b564-ebe0ef11e363\" (UID: \"cde17633-1a9e-4744-b564-ebe0ef11e363\") "
Jul 09 12:44:45.537906 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:45.537925    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde17633-1a9e-4744-b564-ebe0ef11e363-maas-api-tls\") pod \"cde17633-1a9e-4744-b564-ebe0ef11e363\" (UID: \"cde17633-1a9e-4744-b564-ebe0ef11e363\") "
Jul 09 12:44:45.538137 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:45.537963    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9v9w\" (UniqueName: \"kubernetes.io/projected/cde17633-1a9e-4744-b564-ebe0ef11e363-kube-api-access-l9v9w\") pod \"cde17633-1a9e-4744-b564-ebe0ef11e363\" (UID: \"cde17633-1a9e-4744-b564-ebe0ef11e363\") "
Jul 09 12:44:45.538267 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:45.538242    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde17633-1a9e-4744-b564-ebe0ef11e363-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cde17633-1a9e-4744-b564-ebe0ef11e363" (UID: "cde17633-1a9e-4744-b564-ebe0ef11e363"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:44:45.540357 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:45.540326    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde17633-1a9e-4744-b564-ebe0ef11e363-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cde17633-1a9e-4744-b564-ebe0ef11e363" (UID: "cde17633-1a9e-4744-b564-ebe0ef11e363"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:44:45.540357 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:45.540334    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde17633-1a9e-4744-b564-ebe0ef11e363-kube-api-access-l9v9w" (OuterVolumeSpecName: "kube-api-access-l9v9w") pod "cde17633-1a9e-4744-b564-ebe0ef11e363" (UID: "cde17633-1a9e-4744-b564-ebe0ef11e363"). InnerVolumeSpecName "kube-api-access-l9v9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:44:45.638766 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:45.638728    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde17633-1a9e-4744-b564-ebe0ef11e363-openshift-service-ca\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:44:45.638766 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:45.638758    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde17633-1a9e-4744-b564-ebe0ef11e363-maas-api-tls\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:44:45.638766 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:45.638768    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9v9w\" (UniqueName: \"kubernetes.io/projected/cde17633-1a9e-4744-b564-ebe0ef11e363-kube-api-access-l9v9w\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:44:46.327481 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:46.327448    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w" event={"ID":"cde17633-1a9e-4744-b564-ebe0ef11e363","Type":"ContainerDied","Data":"3c6e8e88e51d05c4d52dad70e862af7cdbbf9569bf85586d98bf241f00975bf1"}
Jul 09 12:44:46.327935 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:46.327501    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"
Jul 09 12:44:46.327935 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:46.327513    2565 scope.go:117] "RemoveContainer" containerID="e5cdbd55a665f1c89b16bf0232e01cda3f2b57a8beff7e2d579001f5ad861785"
Jul 09 12:44:46.344174 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:46.344144    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"]
Jul 09 12:44:46.347685 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:46.347655    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4861fa-577f647c9d-4wx8w"]
Jul 09 12:44:47.894489 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:47.894442    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cde17633-1a9e-4744-b564-ebe0ef11e363" path="/var/lib/kubelet/pods/cde17633-1a9e-4744-b564-ebe0ef11e363/volumes"
Jul 09 12:44:48.213503 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:48.213408    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:44:59.269256 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.269191    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79" podUID="41ad6072-5084-4810-9ac1-50503305727d" containerName="main" containerID="cri-o://9ff2b6b030f1b2ed40dd7181761228e6f3c6528dfc0b887b71be07cf34e71264" gracePeriod=45
Jul 09 12:44:59.401722 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.401671    2565 generic.go:356] "Generic (PLEG): container finished" podID="41ad6072-5084-4810-9ac1-50503305727d" containerID="9ff2b6b030f1b2ed40dd7181761228e6f3c6528dfc0b887b71be07cf34e71264" exitCode=0
Jul 09 12:44:59.401888 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.401738    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79" event={"ID":"41ad6072-5084-4810-9ac1-50503305727d","Type":"ContainerDied","Data":"9ff2b6b030f1b2ed40dd7181761228e6f3c6528dfc0b887b71be07cf34e71264"}
Jul 09 12:44:59.526634 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.526567    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:44:59.573576 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.573545    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-tmp-dir\") pod \"41ad6072-5084-4810-9ac1-50503305727d\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") "
Jul 09 12:44:59.573762 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.573635    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d65pj\" (UniqueName: \"kubernetes.io/projected/41ad6072-5084-4810-9ac1-50503305727d-kube-api-access-d65pj\") pod \"41ad6072-5084-4810-9ac1-50503305727d\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") "
Jul 09 12:44:59.573762 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.573738    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-model-cache\") pod \"41ad6072-5084-4810-9ac1-50503305727d\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") "
Jul 09 12:44:59.573864 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.573781    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-home\") pod \"41ad6072-5084-4810-9ac1-50503305727d\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") "
Jul 09 12:44:59.573864 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.573811    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-dshm\") pod \"41ad6072-5084-4810-9ac1-50503305727d\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") "
Jul 09 12:44:59.573864 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.573841    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41ad6072-5084-4810-9ac1-50503305727d-tls-certs\") pod \"41ad6072-5084-4810-9ac1-50503305727d\" (UID: \"41ad6072-5084-4810-9ac1-50503305727d\") "
Jul 09 12:44:59.573864 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.573852    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "41ad6072-5084-4810-9ac1-50503305727d" (UID: "41ad6072-5084-4810-9ac1-50503305727d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:44:59.574101 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.574033    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-model-cache" (OuterVolumeSpecName: "model-cache") pod "41ad6072-5084-4810-9ac1-50503305727d" (UID: "41ad6072-5084-4810-9ac1-50503305727d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:44:59.574101 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.574079    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-home" (OuterVolumeSpecName: "home") pod "41ad6072-5084-4810-9ac1-50503305727d" (UID: "41ad6072-5084-4810-9ac1-50503305727d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:44:59.574203 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.574158    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-model-cache\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:44:59.574203 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.574170    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-home\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:44:59.574203 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.574198    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-tmp-dir\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:44:59.576148 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.576123    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-dshm" (OuterVolumeSpecName: "dshm") pod "41ad6072-5084-4810-9ac1-50503305727d" (UID: "41ad6072-5084-4810-9ac1-50503305727d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 12:44:59.576148 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.576138    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41ad6072-5084-4810-9ac1-50503305727d-kube-api-access-d65pj" (OuterVolumeSpecName: "kube-api-access-d65pj") pod "41ad6072-5084-4810-9ac1-50503305727d" (UID: "41ad6072-5084-4810-9ac1-50503305727d"). InnerVolumeSpecName "kube-api-access-d65pj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:44:59.576314 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.576195    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41ad6072-5084-4810-9ac1-50503305727d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "41ad6072-5084-4810-9ac1-50503305727d" (UID: "41ad6072-5084-4810-9ac1-50503305727d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:44:59.675750 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.675674    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d65pj\" (UniqueName: \"kubernetes.io/projected/41ad6072-5084-4810-9ac1-50503305727d-kube-api-access-d65pj\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:44:59.675750 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.675745    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41ad6072-5084-4810-9ac1-50503305727d-dshm\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:44:59.675750 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:44:59.675755    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41ad6072-5084-4810-9ac1-50503305727d-tls-certs\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:45:00.408349 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:00.408322    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"
Jul 09 12:45:00.408873 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:00.408318    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79" event={"ID":"41ad6072-5084-4810-9ac1-50503305727d","Type":"ContainerDied","Data":"12c464864ade4da5a65b0df1ab2317881df512b980f6f95f692f75b04125f93f"}
Jul 09 12:45:00.408873 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:00.408465    2565 scope.go:117] "RemoveContainer" containerID="9ff2b6b030f1b2ed40dd7181761228e6f3c6528dfc0b887b71be07cf34e71264"
Jul 09 12:45:00.424344 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:00.424317    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"]
Jul 09 12:45:00.426557 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:00.426534    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a2f2c0/test-model-a2f2c0-kserve-58bdf9b445-24x79"]
Jul 09 12:45:01.894800 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:01.894768    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41ad6072-5084-4810-9ac1-50503305727d" path="/var/lib/kubelet/pods/41ad6072-5084-4810-9ac1-50503305727d/volumes"
Jul 09 12:45:28.141791 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.141751    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"]
Jul 09 12:45:28.142311 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.142012    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp" podUID="e6be2862-46db-46cf-8b45-bd371f103a40" containerName="maas-api" containerID="cri-o://914eb88a4678adcc4c03a3673536efa350d90f0698d5918c65b0f3c49a876b81" gracePeriod=30
Jul 09 12:45:28.407404 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.407322    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:45:28.445176 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.445132    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsh9k\" (UniqueName: \"kubernetes.io/projected/e6be2862-46db-46cf-8b45-bd371f103a40-kube-api-access-wsh9k\") pod \"e6be2862-46db-46cf-8b45-bd371f103a40\" (UID: \"e6be2862-46db-46cf-8b45-bd371f103a40\") "
Jul 09 12:45:28.445176 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.445171    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6be2862-46db-46cf-8b45-bd371f103a40-maas-api-tls\") pod \"e6be2862-46db-46cf-8b45-bd371f103a40\" (UID: \"e6be2862-46db-46cf-8b45-bd371f103a40\") "
Jul 09 12:45:28.445439 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.445239    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6be2862-46db-46cf-8b45-bd371f103a40-openshift-service-ca\") pod \"e6be2862-46db-46cf-8b45-bd371f103a40\" (UID: \"e6be2862-46db-46cf-8b45-bd371f103a40\") "
Jul 09 12:45:28.445641 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.445618    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6be2862-46db-46cf-8b45-bd371f103a40-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e6be2862-46db-46cf-8b45-bd371f103a40" (UID: "e6be2862-46db-46cf-8b45-bd371f103a40"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:45:28.447597 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.447557    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6be2862-46db-46cf-8b45-bd371f103a40-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e6be2862-46db-46cf-8b45-bd371f103a40" (UID: "e6be2862-46db-46cf-8b45-bd371f103a40"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:45:28.447766 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.447652    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6be2862-46db-46cf-8b45-bd371f103a40-kube-api-access-wsh9k" (OuterVolumeSpecName: "kube-api-access-wsh9k") pod "e6be2862-46db-46cf-8b45-bd371f103a40" (UID: "e6be2862-46db-46cf-8b45-bd371f103a40"). InnerVolumeSpecName "kube-api-access-wsh9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:45:28.546006 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.545970    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsh9k\" (UniqueName: \"kubernetes.io/projected/e6be2862-46db-46cf-8b45-bd371f103a40-kube-api-access-wsh9k\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:45:28.546006 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.546001    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6be2862-46db-46cf-8b45-bd371f103a40-maas-api-tls\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:45:28.546006 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.546011    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6be2862-46db-46cf-8b45-bd371f103a40-openshift-service-ca\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:45:28.562651 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.562609    2565 generic.go:356] "Generic (PLEG): container finished" podID="e6be2862-46db-46cf-8b45-bd371f103a40" containerID="914eb88a4678adcc4c03a3673536efa350d90f0698d5918c65b0f3c49a876b81" exitCode=0
Jul 09 12:45:28.562651 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.562657    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"
Jul 09 12:45:28.562899 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.562689    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp" event={"ID":"e6be2862-46db-46cf-8b45-bd371f103a40","Type":"ContainerDied","Data":"914eb88a4678adcc4c03a3673536efa350d90f0698d5918c65b0f3c49a876b81"}
Jul 09 12:45:28.562899 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.562779    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp" event={"ID":"e6be2862-46db-46cf-8b45-bd371f103a40","Type":"ContainerDied","Data":"c75830a84c0933ddfc9d86b3a1efdef00176ed3eaa012d35a07f474d26c76c63"}
Jul 09 12:45:28.562899 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.562803    2565 scope.go:117] "RemoveContainer" containerID="914eb88a4678adcc4c03a3673536efa350d90f0698d5918c65b0f3c49a876b81"
Jul 09 12:45:28.572800 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.572773    2565 scope.go:117] "RemoveContainer" containerID="914eb88a4678adcc4c03a3673536efa350d90f0698d5918c65b0f3c49a876b81"
Jul 09 12:45:28.573131 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:45:28.573114    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"914eb88a4678adcc4c03a3673536efa350d90f0698d5918c65b0f3c49a876b81\": container with ID starting with 914eb88a4678adcc4c03a3673536efa350d90f0698d5918c65b0f3c49a876b81 not found: ID does not exist" containerID="914eb88a4678adcc4c03a3673536efa350d90f0698d5918c65b0f3c49a876b81"
Jul 09 12:45:28.573178 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.573141    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"914eb88a4678adcc4c03a3673536efa350d90f0698d5918c65b0f3c49a876b81"} err="failed to get container status \"914eb88a4678adcc4c03a3673536efa350d90f0698d5918c65b0f3c49a876b81\": rpc error: code = NotFound desc = could not find container \"914eb88a4678adcc4c03a3673536efa350d90f0698d5918c65b0f3c49a876b81\": container with ID starting with 914eb88a4678adcc4c03a3673536efa350d90f0698d5918c65b0f3c49a876b81 not found: ID does not exist"
Jul 09 12:45:28.586267 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.586223    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"]
Jul 09 12:45:28.593301 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:28.593267    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a2f2c0-84657fd9f9-hfxnp"]
Jul 09 12:45:29.894831 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:29.894789    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6be2862-46db-46cf-8b45-bd371f103a40" path="/var/lib/kubelet/pods/e6be2862-46db-46cf-8b45-bd371f103a40/volumes"
Jul 09 12:45:31.328153 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:31.328114    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:45:38.629298 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.629260    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"]
Jul 09 12:45:38.629917 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.629815    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6be2862-46db-46cf-8b45-bd371f103a40" containerName="maas-api"
Jul 09 12:45:38.629917 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.629834    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6be2862-46db-46cf-8b45-bd371f103a40" containerName="maas-api"
Jul 09 12:45:38.629917 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.629843    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde17633-1a9e-4744-b564-ebe0ef11e363" containerName="maas-api"
Jul 09 12:45:38.629917 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.629849    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde17633-1a9e-4744-b564-ebe0ef11e363" containerName="maas-api"
Jul 09 12:45:38.629917 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.629865    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ad6072-5084-4810-9ac1-50503305727d" containerName="main"
Jul 09 12:45:38.629917 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.629872    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ad6072-5084-4810-9ac1-50503305727d" containerName="main"
Jul 09 12:45:38.630165 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.629959    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="41ad6072-5084-4810-9ac1-50503305727d" containerName="main"
Jul 09 12:45:38.630165 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.629971    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6be2862-46db-46cf-8b45-bd371f103a40" containerName="maas-api"
Jul 09 12:45:38.630165 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.629983    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="cde17633-1a9e-4744-b564-ebe0ef11e363" containerName="maas-api"
Jul 09 12:45:38.634412 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.634396    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:38.637299 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.637269    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a7e56d-serving-cert\""
Jul 09 12:45:38.644936 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.644907    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"]
Jul 09 12:45:38.751044 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.750997    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g\" (UID: \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:38.751044 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.751047    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lktn6\" (UniqueName: \"kubernetes.io/projected/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-kube-api-access-lktn6\") pod \"maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g\" (UID: \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:38.751270 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.751122    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-maas-api-tls\") pod \"maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g\" (UID: \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:38.851861 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.851821    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g\" (UID: \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:38.851861 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.851864    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lktn6\" (UniqueName: \"kubernetes.io/projected/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-kube-api-access-lktn6\") pod \"maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g\" (UID: \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:38.852123 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.851917    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-maas-api-tls\") pod \"maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g\" (UID: \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:38.852123 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:45:38.852033    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-serving-cert: secret "maas-api-e2e-api-a-a7e56d-serving-cert" not found
Jul 09 12:45:38.852123 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:45:38.852102    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-maas-api-tls podName:a9f9c809-ab67-49cc-9a45-e66a40a33e6c nodeName:}" failed. No retries permitted until 2026-07-09 12:45:39.35208707 +0000 UTC m=+2612.029616105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-maas-api-tls") pod "maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g" (UID: "a9f9c809-ab67-49cc-9a45-e66a40a33e6c") : secret "maas-api-e2e-api-a-a7e56d-serving-cert" not found
Jul 09 12:45:38.852497 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.852476    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-openshift-service-ca\") pod \"maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g\" (UID: \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:38.861744 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:38.861720    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lktn6\" (UniqueName: \"kubernetes.io/projected/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-kube-api-access-lktn6\") pod \"maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g\" (UID: \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:39.356632 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:39.356588    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-maas-api-tls\") pod \"maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g\" (UID: \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:39.359364 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:39.359335    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-maas-api-tls\") pod \"maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g\" (UID: \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:39.545871 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:39.545840    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:39.689689 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:39.689655    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"]
Jul 09 12:45:39.691772 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:45:39.691743    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9f9c809_ab67_49cc_9a45_e66a40a33e6c.slice/crio-4f96f9f219b785f455cbc73f5c20ad03d29c7803687ab29978b7a6da6e744995 WatchSource:0}: Error finding container 4f96f9f219b785f455cbc73f5c20ad03d29c7803687ab29978b7a6da6e744995: Status 404 returned error can't find the container with id 4f96f9f219b785f455cbc73f5c20ad03d29c7803687ab29978b7a6da6e744995
Jul 09 12:45:40.632209 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:40.632171    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g" event={"ID":"a9f9c809-ab67-49cc-9a45-e66a40a33e6c","Type":"ContainerStarted","Data":"af12b566cb7bb1b25e1a641f2e579141677fb6445552996e29ec6d549e92d230"}
Jul 09 12:45:40.632439 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:40.632235    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g" event={"ID":"a9f9c809-ab67-49cc-9a45-e66a40a33e6c","Type":"ContainerStarted","Data":"4f96f9f219b785f455cbc73f5c20ad03d29c7803687ab29978b7a6da6e744995"}
Jul 09 12:45:40.632439 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:40.632279    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:40.649813 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:40.649758    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g" podStartSLOduration=2.238285139 podStartE2EDuration="2.649739762s" totalImagesPullingTime="411.454623ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:45:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:45:40.647551005 +0000 UTC m=+2613.325080067" watchObservedRunningTime="2026-07-09 12:45:40.649739762 +0000 UTC m=+2613.327268832"
Jul 09 12:45:46.022519 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:46.022479    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"]
Jul 09 12:45:46.026460 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:46.026435    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:46.029465 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:46.029426    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-bd2897-serving-cert\""
Jul 09 12:45:46.035863 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:46.035834    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"]
Jul 09 12:45:46.123365 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:46.123324    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-openshift-service-ca\") pod \"maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s\" (UID: \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:46.123562 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:46.123401    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-maas-api-tls\") pod \"maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s\" (UID: \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:46.123562 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:46.123430    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghkq5\" (UniqueName: \"kubernetes.io/projected/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-kube-api-access-ghkq5\") pod \"maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s\" (UID: \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:46.224426 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:46.224379    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-openshift-service-ca\") pod \"maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s\" (UID: \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:46.224641 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:46.224461    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-maas-api-tls\") pod \"maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s\" (UID: \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:46.224641 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:46.224487    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghkq5\" (UniqueName: \"kubernetes.io/projected/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-kube-api-access-ghkq5\") pod \"maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s\" (UID: \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:46.224641 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:45:46.224603    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-serving-cert: secret "maas-api-e2e-api-b-bd2897-serving-cert" not found
Jul 09 12:45:46.224830 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:45:46.224685    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-maas-api-tls podName:1cfde2ed-0ab0-483b-9614-5bb342e1b23c nodeName:}" failed. No retries permitted until 2026-07-09 12:45:46.72466934 +0000 UTC m=+2619.402198380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-maas-api-tls") pod "maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s" (UID: "1cfde2ed-0ab0-483b-9614-5bb342e1b23c") : secret "maas-api-e2e-api-b-bd2897-serving-cert" not found
Jul 09 12:45:46.225127 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:46.225108    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-openshift-service-ca\") pod \"maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s\" (UID: \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:46.233645 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:46.233614    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghkq5\" (UniqueName: \"kubernetes.io/projected/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-kube-api-access-ghkq5\") pod \"maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s\" (UID: \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:46.643796 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:46.643765    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:46.728902 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:46.728856    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-maas-api-tls\") pod \"maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s\" (UID: \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:46.729114 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:45:46.728993    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-serving-cert: secret "maas-api-e2e-api-b-bd2897-serving-cert" not found
Jul 09 12:45:46.729114 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:45:46.729080    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-maas-api-tls podName:1cfde2ed-0ab0-483b-9614-5bb342e1b23c nodeName:}" failed. No retries permitted until 2026-07-09 12:45:47.729047087 +0000 UTC m=+2620.406576124 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-maas-api-tls") pod "maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s" (UID: "1cfde2ed-0ab0-483b-9614-5bb342e1b23c") : secret "maas-api-e2e-api-b-bd2897-serving-cert" not found
Jul 09 12:45:47.228203 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:47.228152    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:45:47.738182 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:47.738143    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-maas-api-tls\") pod \"maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s\" (UID: \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:47.740821 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:47.740788    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-maas-api-tls\") pod \"maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s\" (UID: \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:47.839357 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:47.839309    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:47.989934 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:47.989903    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"]
Jul 09 12:45:47.990786 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:45:47.990766    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice/crio-391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a WatchSource:0}: Error finding container 391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a: Status 404 returned error can't find the container with id 391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a
Jul 09 12:45:48.678871 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:48.678770    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s" event={"ID":"1cfde2ed-0ab0-483b-9614-5bb342e1b23c","Type":"ContainerStarted","Data":"e074acb13ccbc09ce6d1d1f616da7b63e25f56a7be4cd24fd50b1e2b2b1e1c96"}
Jul 09 12:45:48.678871 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:48.678843    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:48.678871 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:48.678865    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s" event={"ID":"1cfde2ed-0ab0-483b-9614-5bb342e1b23c","Type":"ContainerStarted","Data":"391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a"}
Jul 09 12:45:48.697560 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:48.697499    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s" podStartSLOduration=3.323584373 podStartE2EDuration="3.697482118s" totalImagesPullingTime="373.897745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:45:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:45:48.696145261 +0000 UTC m=+2621.373674320" watchObservedRunningTime="2026-07-09 12:45:48.697482118 +0000 UTC m=+2621.375011195"
Jul 09 12:45:54.691200 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:54.691169    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:45:59.354272 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.354237    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"]
Jul 09 12:45:59.354746 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.354442    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g" podUID="a9f9c809-ab67-49cc-9a45-e66a40a33e6c" containerName="maas-api" containerID="cri-o://af12b566cb7bb1b25e1a641f2e579141677fb6445552996e29ec6d549e92d230" gracePeriod=30
Jul 09 12:45:59.612272 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.612206    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:59.739859 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.739825    2565 generic.go:356] "Generic (PLEG): container finished" podID="a9f9c809-ab67-49cc-9a45-e66a40a33e6c" containerID="af12b566cb7bb1b25e1a641f2e579141677fb6445552996e29ec6d549e92d230" exitCode=0
Jul 09 12:45:59.740054 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.739873    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"
Jul 09 12:45:59.740054 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.739908    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g" event={"ID":"a9f9c809-ab67-49cc-9a45-e66a40a33e6c","Type":"ContainerDied","Data":"af12b566cb7bb1b25e1a641f2e579141677fb6445552996e29ec6d549e92d230"}
Jul 09 12:45:59.740054 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.739960    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g" event={"ID":"a9f9c809-ab67-49cc-9a45-e66a40a33e6c","Type":"ContainerDied","Data":"4f96f9f219b785f455cbc73f5c20ad03d29c7803687ab29978b7a6da6e744995"}
Jul 09 12:45:59.740054 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.739983    2565 scope.go:117] "RemoveContainer" containerID="af12b566cb7bb1b25e1a641f2e579141677fb6445552996e29ec6d549e92d230"
Jul 09 12:45:59.748994 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.748976    2565 scope.go:117] "RemoveContainer" containerID="af12b566cb7bb1b25e1a641f2e579141677fb6445552996e29ec6d549e92d230"
Jul 09 12:45:59.749343 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:45:59.749317    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af12b566cb7bb1b25e1a641f2e579141677fb6445552996e29ec6d549e92d230\": container with ID starting with af12b566cb7bb1b25e1a641f2e579141677fb6445552996e29ec6d549e92d230 not found: ID does not exist" containerID="af12b566cb7bb1b25e1a641f2e579141677fb6445552996e29ec6d549e92d230"
Jul 09 12:45:59.749446 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.749350    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af12b566cb7bb1b25e1a641f2e579141677fb6445552996e29ec6d549e92d230"} err="failed to get container status \"af12b566cb7bb1b25e1a641f2e579141677fb6445552996e29ec6d549e92d230\": rpc error: code = NotFound desc = could not find container \"af12b566cb7bb1b25e1a641f2e579141677fb6445552996e29ec6d549e92d230\": container with ID starting with af12b566cb7bb1b25e1a641f2e579141677fb6445552996e29ec6d549e92d230 not found: ID does not exist"
Jul 09 12:45:59.749551 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.749535    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lktn6\" (UniqueName: \"kubernetes.io/projected/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-kube-api-access-lktn6\") pod \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\" (UID: \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\") "
Jul 09 12:45:59.749621 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.749596    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-maas-api-tls\") pod \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\" (UID: \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\") "
Jul 09 12:45:59.749752 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.749732    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-openshift-service-ca\") pod \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\" (UID: \"a9f9c809-ab67-49cc-9a45-e66a40a33e6c\") "
Jul 09 12:45:59.750060 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.750034    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a9f9c809-ab67-49cc-9a45-e66a40a33e6c" (UID: "a9f9c809-ab67-49cc-9a45-e66a40a33e6c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:45:59.751797 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.751776    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-kube-api-access-lktn6" (OuterVolumeSpecName: "kube-api-access-lktn6") pod "a9f9c809-ab67-49cc-9a45-e66a40a33e6c" (UID: "a9f9c809-ab67-49cc-9a45-e66a40a33e6c"). InnerVolumeSpecName "kube-api-access-lktn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:45:59.751878 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.751842    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a9f9c809-ab67-49cc-9a45-e66a40a33e6c" (UID: "a9f9c809-ab67-49cc-9a45-e66a40a33e6c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:45:59.850852 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.850809    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lktn6\" (UniqueName: \"kubernetes.io/projected/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-kube-api-access-lktn6\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:45:59.850852 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.850847    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-maas-api-tls\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:45:59.850852 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:45:59.850859    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9f9c809-ab67-49cc-9a45-e66a40a33e6c-openshift-service-ca\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:46:00.056357 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:00.056321    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"]
Jul 09 12:46:00.059514 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:00.059488    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a7e56d-7db9ffb569-sn27g"]
Jul 09 12:46:01.894741 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:01.894689    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9f9c809-ab67-49cc-9a45-e66a40a33e6c" path="/var/lib/kubelet/pods/a9f9c809-ab67-49cc-9a45-e66a40a33e6c/volumes"
Jul 09 12:46:03.323450 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:03.323416    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:46:06.536464 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:06.536427    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"]
Jul 09 12:46:06.536936 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:06.536644    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s" podUID="1cfde2ed-0ab0-483b-9614-5bb342e1b23c" containerName="maas-api" containerID="cri-o://e074acb13ccbc09ce6d1d1f616da7b63e25f56a7be4cd24fd50b1e2b2b1e1c96" gracePeriod=30
Jul 09 12:46:06.781019 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:06.780986    2565 generic.go:356] "Generic (PLEG): container finished" podID="1cfde2ed-0ab0-483b-9614-5bb342e1b23c" containerID="e074acb13ccbc09ce6d1d1f616da7b63e25f56a7be4cd24fd50b1e2b2b1e1c96" exitCode=0
Jul 09 12:46:06.781173 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:06.781066    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s" event={"ID":"1cfde2ed-0ab0-483b-9614-5bb342e1b23c","Type":"ContainerDied","Data":"e074acb13ccbc09ce6d1d1f616da7b63e25f56a7be4cd24fd50b1e2b2b1e1c96"}
Jul 09 12:46:06.781173 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:06.781122    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s" event={"ID":"1cfde2ed-0ab0-483b-9614-5bb342e1b23c","Type":"ContainerDied","Data":"391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a"}
Jul 09 12:46:06.781173 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:06.781150    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a"
Jul 09 12:46:06.794356 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:06.794292    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:46:06.919430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:06.919389    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghkq5\" (UniqueName: \"kubernetes.io/projected/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-kube-api-access-ghkq5\") pod \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\" (UID: \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\") "
Jul 09 12:46:06.919614 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:06.919478    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-maas-api-tls\") pod \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\" (UID: \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\") "
Jul 09 12:46:06.919614 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:06.919511    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-openshift-service-ca\") pod \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\" (UID: \"1cfde2ed-0ab0-483b-9614-5bb342e1b23c\") "
Jul 09 12:46:06.919960 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:06.919927    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1cfde2ed-0ab0-483b-9614-5bb342e1b23c" (UID: "1cfde2ed-0ab0-483b-9614-5bb342e1b23c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:46:06.921748 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:06.921717    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-kube-api-access-ghkq5" (OuterVolumeSpecName: "kube-api-access-ghkq5") pod "1cfde2ed-0ab0-483b-9614-5bb342e1b23c" (UID: "1cfde2ed-0ab0-483b-9614-5bb342e1b23c"). InnerVolumeSpecName "kube-api-access-ghkq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:46:06.921748 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:06.921721    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1cfde2ed-0ab0-483b-9614-5bb342e1b23c" (UID: "1cfde2ed-0ab0-483b-9614-5bb342e1b23c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:46:07.023763 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:07.021429    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-maas-api-tls\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:46:07.023763 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:07.021475    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-openshift-service-ca\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:46:07.023763 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:07.021497    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghkq5\" (UniqueName: \"kubernetes.io/projected/1cfde2ed-0ab0-483b-9614-5bb342e1b23c-kube-api-access-ghkq5\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:46:07.787035 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:07.787000    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"
Jul 09 12:46:07.812365 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:07.812328    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"]
Jul 09 12:46:07.815487 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:07.815462    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-bd2897-6b4747bd96-v5q7s"]
Jul 09 12:46:07.894404 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:07.894373    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cfde2ed-0ab0-483b-9614-5bb342e1b23c" path="/var/lib/kubelet/pods/1cfde2ed-0ab0-483b-9614-5bb342e1b23c/volumes"
Jul 09 12:46:15.688547 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:46:15.688503    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice/crio-391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice\": RecentStats: unable to find data in memory cache]"
Jul 09 12:46:16.112227 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:46:16.112187    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice/crio-391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a\": RecentStats: unable to find data in memory cache]"
Jul 09 12:46:16.151564 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.151524    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"]
Jul 09 12:46:16.152058 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.152032    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9f9c809-ab67-49cc-9a45-e66a40a33e6c" containerName="maas-api"
Jul 09 12:46:16.152058 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.152056    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9f9c809-ab67-49cc-9a45-e66a40a33e6c" containerName="maas-api"
Jul 09 12:46:16.152237 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.152076    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cfde2ed-0ab0-483b-9614-5bb342e1b23c" containerName="maas-api"
Jul 09 12:46:16.152237 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.152085    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cfde2ed-0ab0-483b-9614-5bb342e1b23c" containerName="maas-api"
Jul 09 12:46:16.152237 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.152192    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cfde2ed-0ab0-483b-9614-5bb342e1b23c" containerName="maas-api"
Jul 09 12:46:16.152237 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.152206    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9f9c809-ab67-49cc-9a45-e66a40a33e6c" containerName="maas-api"
Jul 09 12:46:16.155430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.155408    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:46:16.158148 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.158126    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-e28f57-serving-cert\""
Jul 09 12:46:16.168290 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.168258    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"]
Jul 09 12:46:16.194270 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.194232    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn\" (UID: \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:46:16.194447 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.194289    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4fr2\" (UniqueName: \"kubernetes.io/projected/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-kube-api-access-c4fr2\") pod \"maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn\" (UID: \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:46:16.194447 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.194350    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-maas-api-tls\") pod \"maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn\" (UID: \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:46:16.295262 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.295220    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn\" (UID: \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:46:16.295442 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.295282    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4fr2\" (UniqueName: \"kubernetes.io/projected/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-kube-api-access-c4fr2\") pod \"maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn\" (UID: \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:46:16.295442 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.295339    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-maas-api-tls\") pod \"maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn\" (UID: \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:46:16.295442 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:46:16.295439    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-serving-cert: secret "maas-api-e2e-shared-a-e28f57-serving-cert" not found
Jul 09 12:46:16.295551 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:46:16.295496    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-maas-api-tls podName:ec3b98ba-a4f9-4b32-bc7b-4798890c4443 nodeName:}" failed. No retries permitted until 2026-07-09 12:46:16.795479906 +0000 UTC m=+2649.473008942 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-maas-api-tls") pod "maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn" (UID: "ec3b98ba-a4f9-4b32-bc7b-4798890c4443") : secret "maas-api-e2e-shared-a-e28f57-serving-cert" not found
Jul 09 12:46:16.296035 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.296014    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn\" (UID: \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:46:16.305049 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.305020    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4fr2\" (UniqueName: \"kubernetes.io/projected/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-kube-api-access-c4fr2\") pod \"maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn\" (UID: \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:46:16.799731 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.799664    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-maas-api-tls\") pod \"maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn\" (UID: \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:46:16.802398 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:16.802371    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-maas-api-tls\") pod \"maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn\" (UID: \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:46:17.067415 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:17.067377    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:46:17.220548 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:17.220520    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"]
Jul 09 12:46:17.222388 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:46:17.222359    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec3b98ba_a4f9_4b32_bc7b_4798890c4443.slice/crio-de98f4601ef4b0cf7b41fbc4947dc8566faa2ba28cbacf61beae99f44457e14c WatchSource:0}: Error finding container de98f4601ef4b0cf7b41fbc4947dc8566faa2ba28cbacf61beae99f44457e14c: Status 404 returned error can't find the container with id de98f4601ef4b0cf7b41fbc4947dc8566faa2ba28cbacf61beae99f44457e14c
Jul 09 12:46:17.850504 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:17.850471    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn" event={"ID":"ec3b98ba-a4f9-4b32-bc7b-4798890c4443","Type":"ContainerStarted","Data":"de98f4601ef4b0cf7b41fbc4947dc8566faa2ba28cbacf61beae99f44457e14c"}
Jul 09 12:46:18.026273 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:18.026236    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:46:18.857625 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:18.857572    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn" event={"ID":"ec3b98ba-a4f9-4b32-bc7b-4798890c4443","Type":"ContainerStarted","Data":"e4606763aeae7ee609f1349efbff5ff51d92cc898ee0af5339aebd63bff13c9f"}
Jul 09 12:46:18.858112 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:18.857644    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:46:18.876941 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:18.876883    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn" podStartSLOduration=2.309887173 podStartE2EDuration="2.876866092s" totalImagesPullingTime="566.978919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:46:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:46:18.874684546 +0000 UTC m=+2651.552213604" watchObservedRunningTime="2026-07-09 12:46:18.876866092 +0000 UTC m=+2651.554395183"
Jul 09 12:46:24.532136 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:24.532093    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"]
Jul 09 12:46:24.536722 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:24.536669    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:46:24.542319 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:24.542295    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-e490e6-serving-cert\""
Jul 09 12:46:24.550165 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:24.550140    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"]
Jul 09 12:46:24.574963 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:24.574921    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlg87\" (UniqueName: \"kubernetes.io/projected/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-kube-api-access-tlg87\") pod \"maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx\" (UID: \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:46:24.575131 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:24.574973    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-maas-api-tls\") pod \"maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx\" (UID: \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:46:24.575131 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:24.575038    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx\" (UID: \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:46:24.675783 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:24.675738    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx\" (UID: \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:46:24.675974 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:24.675880    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlg87\" (UniqueName: \"kubernetes.io/projected/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-kube-api-access-tlg87\") pod \"maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx\" (UID: \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:46:24.675974 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:24.675910    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-maas-api-tls\") pod \"maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx\" (UID: \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:46:24.676048 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:46:24.676027    2565 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-serving-cert: secret "maas-api-e2e-shared-b-e490e6-serving-cert" not found
Jul 09 12:46:24.676108 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:46:24.676097    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-maas-api-tls podName:7294ccdc-2d00-47fc-8dc0-42f7a8c304df nodeName:}" failed. No retries permitted until 2026-07-09 12:46:25.176076767 +0000 UTC m=+2657.853605807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-maas-api-tls") pod "maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx" (UID: "7294ccdc-2d00-47fc-8dc0-42f7a8c304df") : secret "maas-api-e2e-shared-b-e490e6-serving-cert" not found
Jul 09 12:46:24.676399 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:24.676379    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx\" (UID: \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:46:24.687181 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:24.687151    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlg87\" (UniqueName: \"kubernetes.io/projected/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-kube-api-access-tlg87\") pod \"maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx\" (UID: \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:46:24.869475 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:24.869445    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:46:25.180342 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:25.180245    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-maas-api-tls\") pod \"maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx\" (UID: \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:46:25.183026 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:25.182989    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-maas-api-tls\") pod \"maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx\" (UID: \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:46:25.447826 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:25.447730    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:46:25.587393 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:25.587365    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"]
Jul 09 12:46:25.588122 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:46:25.588091    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7294ccdc_2d00_47fc_8dc0_42f7a8c304df.slice/crio-f2eb80d73219fbaf454cbfa560e7997a9a0cdf65a054920113907ad43c1789c6 WatchSource:0}: Error finding container f2eb80d73219fbaf454cbfa560e7997a9a0cdf65a054920113907ad43c1789c6: Status 404 returned error can't find the container with id f2eb80d73219fbaf454cbfa560e7997a9a0cdf65a054920113907ad43c1789c6
Jul 09 12:46:25.896502 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:25.896459    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx" event={"ID":"7294ccdc-2d00-47fc-8dc0-42f7a8c304df","Type":"ContainerStarted","Data":"f2eb80d73219fbaf454cbfa560e7997a9a0cdf65a054920113907ad43c1789c6"}
Jul 09 12:46:26.161617 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:46:26.161523    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice/crio-391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a\": RecentStats: unable to find data in memory cache]"
Jul 09 12:46:26.903009 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:26.902967    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx" event={"ID":"7294ccdc-2d00-47fc-8dc0-42f7a8c304df","Type":"ContainerStarted","Data":"791800888d6e3a9882ff62f901348a629d7a425bb8b42381f5fab49181f47c30"}
Jul 09 12:46:26.903517 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:26.903101    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:46:26.927610 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:26.927555    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx" podStartSLOduration=2.553641372 podStartE2EDuration="2.927538289s" totalImagesPullingTime="373.896917ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 12:46:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:46:26.925907934 +0000 UTC m=+2659.603436992" watchObservedRunningTime="2026-07-09 12:46:26.927538289 +0000 UTC m=+2659.605067346"
Jul 09 12:46:28.142551 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:28.142511    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:46:30.653251 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:46:30.653209    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice/crio-391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice\": RecentStats: unable to find data in memory cache]"
Jul 09 12:46:32.914753 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:46:32.914718    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:46:36.209475 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:46:36.209422    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice/crio-391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a\": RecentStats: unable to find data in memory cache]"
Jul 09 12:46:45.693134 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:46:45.693092    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice/crio-391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a\": RecentStats: unable to find data in memory cache]"
Jul 09 12:46:46.220467 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:46:46.220422    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice/crio-391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a\": RecentStats: unable to find data in memory cache]"
Jul 09 12:46:56.268287 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:46:56.268245    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice/crio-391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice\": RecentStats: unable to find data in memory cache]"
Jul 09 12:46:58.124889 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:46:58.124842    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice/crio-391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice\": RecentStats: unable to find data in memory cache]"
Jul 09 12:46:58.125474 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:46:58.125018    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice/crio-391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice\": RecentStats: unable to find data in memory cache]"
Jul 09 12:47:00.647793 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:47:00.647665    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice/crio-391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice\": RecentStats: unable to find data in memory cache]"
Jul 09 12:47:06.278558 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:47:06.278506    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice/crio-391c48962161597b5f6376b582a487139122e3ef96ebb4f188b7e3953540c95a\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfde2ed_0ab0_483b_9614_5bb342e1b23c.slice\": RecentStats: unable to find data in memory cache]"
Jul 09 12:47:08.071558 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:47:08.071527    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:47:08.083576 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:47:08.083550    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l2hg7_6b958117-535b-49f3-b74c-4d809655d7d1/ovn-acl-logging/0.log"
Jul 09 12:49:34.341522 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.341482    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"]
Jul 09 12:49:34.344206 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.341742    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn" podUID="ec3b98ba-a4f9-4b32-bc7b-4798890c4443" containerName="maas-api" containerID="cri-o://e4606763aeae7ee609f1349efbff5ff51d92cc898ee0af5339aebd63bff13c9f" gracePeriod=30
Jul 09 12:49:34.600646 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.600555    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:49:34.718861 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.718821    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-openshift-service-ca\") pod \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\" (UID: \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\") "
Jul 09 12:49:34.719039 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.718898    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-maas-api-tls\") pod \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\" (UID: \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\") "
Jul 09 12:49:34.719039 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.718995    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4fr2\" (UniqueName: \"kubernetes.io/projected/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-kube-api-access-c4fr2\") pod \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\" (UID: \"ec3b98ba-a4f9-4b32-bc7b-4798890c4443\") "
Jul 09 12:49:34.719285 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.719253    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ec3b98ba-a4f9-4b32-bc7b-4798890c4443" (UID: "ec3b98ba-a4f9-4b32-bc7b-4798890c4443"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:49:34.721181 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.721147    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ec3b98ba-a4f9-4b32-bc7b-4798890c4443" (UID: "ec3b98ba-a4f9-4b32-bc7b-4798890c4443"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:49:34.721275 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.721207    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-kube-api-access-c4fr2" (OuterVolumeSpecName: "kube-api-access-c4fr2") pod "ec3b98ba-a4f9-4b32-bc7b-4798890c4443" (UID: "ec3b98ba-a4f9-4b32-bc7b-4798890c4443"). InnerVolumeSpecName "kube-api-access-c4fr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:49:34.819897 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.819856    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-maas-api-tls\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:49:34.819897 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.819889    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4fr2\" (UniqueName: \"kubernetes.io/projected/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-kube-api-access-c4fr2\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:49:34.819897 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.819899    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec3b98ba-a4f9-4b32-bc7b-4798890c4443-openshift-service-ca\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:49:34.938594 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.938506    2565 generic.go:356] "Generic (PLEG): container finished" podID="ec3b98ba-a4f9-4b32-bc7b-4798890c4443" containerID="e4606763aeae7ee609f1349efbff5ff51d92cc898ee0af5339aebd63bff13c9f" exitCode=0
Jul 09 12:49:34.938594 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.938554    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn" event={"ID":"ec3b98ba-a4f9-4b32-bc7b-4798890c4443","Type":"ContainerDied","Data":"e4606763aeae7ee609f1349efbff5ff51d92cc898ee0af5339aebd63bff13c9f"}
Jul 09 12:49:34.938594 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.938593    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn" event={"ID":"ec3b98ba-a4f9-4b32-bc7b-4798890c4443","Type":"ContainerDied","Data":"de98f4601ef4b0cf7b41fbc4947dc8566faa2ba28cbacf61beae99f44457e14c"}
Jul 09 12:49:34.938856 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.938616    2565 scope.go:117] "RemoveContainer" containerID="e4606763aeae7ee609f1349efbff5ff51d92cc898ee0af5339aebd63bff13c9f"
Jul 09 12:49:34.938856 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.938557    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"
Jul 09 12:49:34.948369 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.948348    2565 scope.go:117] "RemoveContainer" containerID="e4606763aeae7ee609f1349efbff5ff51d92cc898ee0af5339aebd63bff13c9f"
Jul 09 12:49:34.948742 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:49:34.948677    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4606763aeae7ee609f1349efbff5ff51d92cc898ee0af5339aebd63bff13c9f\": container with ID starting with e4606763aeae7ee609f1349efbff5ff51d92cc898ee0af5339aebd63bff13c9f not found: ID does not exist" containerID="e4606763aeae7ee609f1349efbff5ff51d92cc898ee0af5339aebd63bff13c9f"
Jul 09 12:49:34.948819 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.948733    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4606763aeae7ee609f1349efbff5ff51d92cc898ee0af5339aebd63bff13c9f"} err="failed to get container status \"e4606763aeae7ee609f1349efbff5ff51d92cc898ee0af5339aebd63bff13c9f\": rpc error: code = NotFound desc = could not find container \"e4606763aeae7ee609f1349efbff5ff51d92cc898ee0af5339aebd63bff13c9f\": container with ID starting with e4606763aeae7ee609f1349efbff5ff51d92cc898ee0af5339aebd63bff13c9f not found: ID does not exist"
Jul 09 12:49:34.960228 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.960193    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"]
Jul 09 12:49:34.963775 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:34.963743    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e28f57-85d65c6c95-5frwn"]
Jul 09 12:49:35.894404 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:35.894370    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec3b98ba-a4f9-4b32-bc7b-4798890c4443" path="/var/lib/kubelet/pods/ec3b98ba-a4f9-4b32-bc7b-4798890c4443/volumes"
Jul 09 12:49:37.518693 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:37.518654    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:49:41.928370 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:41.928335    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"]
Jul 09 12:49:41.928895 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:41.928557    2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx" podUID="7294ccdc-2d00-47fc-8dc0-42f7a8c304df" containerName="maas-api" containerID="cri-o://791800888d6e3a9882ff62f901348a629d7a425bb8b42381f5fab49181f47c30" gracePeriod=30
Jul 09 12:49:42.199688 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.199657    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:49:42.390226 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.390192    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-maas-api-tls\") pod \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\" (UID: \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\") "
Jul 09 12:49:42.390405 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.390297    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-openshift-service-ca\") pod \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\" (UID: \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\") "
Jul 09 12:49:42.390405 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.390319    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlg87\" (UniqueName: \"kubernetes.io/projected/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-kube-api-access-tlg87\") pod \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\" (UID: \"7294ccdc-2d00-47fc-8dc0-42f7a8c304df\") "
Jul 09 12:49:42.390677 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.390639    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7294ccdc-2d00-47fc-8dc0-42f7a8c304df" (UID: "7294ccdc-2d00-47fc-8dc0-42f7a8c304df"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 12:49:42.392357 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.392337    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7294ccdc-2d00-47fc-8dc0-42f7a8c304df" (UID: "7294ccdc-2d00-47fc-8dc0-42f7a8c304df"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 12:49:42.392518 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.392496    2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-kube-api-access-tlg87" (OuterVolumeSpecName: "kube-api-access-tlg87") pod "7294ccdc-2d00-47fc-8dc0-42f7a8c304df" (UID: "7294ccdc-2d00-47fc-8dc0-42f7a8c304df"). InnerVolumeSpecName "kube-api-access-tlg87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 12:49:42.491497 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.491389    2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-openshift-service-ca\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:49:42.491497 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.491429    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlg87\" (UniqueName: \"kubernetes.io/projected/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-kube-api-access-tlg87\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:49:42.491497 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.491442    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7294ccdc-2d00-47fc-8dc0-42f7a8c304df-maas-api-tls\") on node \"ip-10-0-141-94.ec2.internal\" DevicePath \"\""
Jul 09 12:49:42.985190 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.985144    2565 generic.go:356] "Generic (PLEG): container finished" podID="7294ccdc-2d00-47fc-8dc0-42f7a8c304df" containerID="791800888d6e3a9882ff62f901348a629d7a425bb8b42381f5fab49181f47c30" exitCode=0
Jul 09 12:49:42.985190 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.985197    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"
Jul 09 12:49:42.985729 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.985234    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx" event={"ID":"7294ccdc-2d00-47fc-8dc0-42f7a8c304df","Type":"ContainerDied","Data":"791800888d6e3a9882ff62f901348a629d7a425bb8b42381f5fab49181f47c30"}
Jul 09 12:49:42.985729 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.985305    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx" event={"ID":"7294ccdc-2d00-47fc-8dc0-42f7a8c304df","Type":"ContainerDied","Data":"f2eb80d73219fbaf454cbfa560e7997a9a0cdf65a054920113907ad43c1789c6"}
Jul 09 12:49:42.985729 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.985341    2565 scope.go:117] "RemoveContainer" containerID="791800888d6e3a9882ff62f901348a629d7a425bb8b42381f5fab49181f47c30"
Jul 09 12:49:42.995198 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.995176    2565 scope.go:117] "RemoveContainer" containerID="791800888d6e3a9882ff62f901348a629d7a425bb8b42381f5fab49181f47c30"
Jul 09 12:49:42.995539 ip-10-0-141-94 kubenswrapper[2565]: E0709 12:49:42.995519    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"791800888d6e3a9882ff62f901348a629d7a425bb8b42381f5fab49181f47c30\": container with ID starting with 791800888d6e3a9882ff62f901348a629d7a425bb8b42381f5fab49181f47c30 not found: ID does not exist" containerID="791800888d6e3a9882ff62f901348a629d7a425bb8b42381f5fab49181f47c30"
Jul 09 12:49:42.995597 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:42.995551    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"791800888d6e3a9882ff62f901348a629d7a425bb8b42381f5fab49181f47c30"} err="failed to get container status \"791800888d6e3a9882ff62f901348a629d7a425bb8b42381f5fab49181f47c30\": rpc error: code = NotFound desc = could not find container \"791800888d6e3a9882ff62f901348a629d7a425bb8b42381f5fab49181f47c30\": container with ID starting with 791800888d6e3a9882ff62f901348a629d7a425bb8b42381f5fab49181f47c30 not found: ID does not exist"
Jul 09 12:49:43.008822 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:43.008789    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"]
Jul 09 12:49:43.016612 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:43.016572    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e490e6-df44d4bd8-mfjkx"]
Jul 09 12:49:43.893961 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:43.893926    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7294ccdc-2d00-47fc-8dc0-42f7a8c304df" path="/var/lib/kubelet/pods/7294ccdc-2d00-47fc-8dc0-42f7a8c304df/volumes"
Jul 09 12:49:50.823438 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:50.823403    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-v4cbc"]
Jul 09 12:49:53.197497 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:53.197465    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jbkjg_4e58dcac-ffcf-4e3f-a36f-83591970b8c4/manager/0.log"
Jul 09 12:49:53.884664 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:53.884616    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd7989cf5-5d2r2_4a0bbd67-ef5e-47c1-8f29-cde9280de159/copy-manifests/0.log"
Jul 09 12:49:53.891289 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:53.891263    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd7989cf5-5d2r2_4a0bbd67-ef5e-47c1-8f29-cde9280de159/manager/0.log"
Jul 09 12:49:54.767485 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:54.767457    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl_7cd144f5-bf25-4752-8353-3c549037a44a/util/0.log"
Jul 09 12:49:54.774567 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:54.774548    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl_7cd144f5-bf25-4752-8353-3c549037a44a/pull/0.log"
Jul 09 12:49:54.781029 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:54.781011    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl_7cd144f5-bf25-4752-8353-3c549037a44a/extract/0.log"
Jul 09 12:49:54.890118 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:54.890090    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5_6efa932f-daab-4dfa-98af-eb221866d49f/util/0.log"
Jul 09 12:49:54.896927 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:54.896908    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5_6efa932f-daab-4dfa-98af-eb221866d49f/pull/0.log"
Jul 09 12:49:54.903446 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:54.903428    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5_6efa932f-daab-4dfa-98af-eb221866d49f/extract/0.log"
Jul 09 12:49:55.012152 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:55.012120    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6_949368a2-d606-447f-83c9-e1a9cfb5798e/util/0.log"
Jul 09 12:49:55.019001 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:55.018924    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6_949368a2-d606-447f-83c9-e1a9cfb5798e/pull/0.log"
Jul 09 12:49:55.025132 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:55.025106    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6_949368a2-d606-447f-83c9-e1a9cfb5798e/extract/0.log"
Jul 09 12:49:55.140958 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:55.139019    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw_9d0ebd1d-d802-492e-93d4-340a5574b3c9/extract/0.log"
Jul 09 12:49:55.145693 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:55.145668    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw_9d0ebd1d-d802-492e-93d4-340a5574b3c9/util/0.log"
Jul 09 12:49:55.152102 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:55.152076    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw_9d0ebd1d-d802-492e-93d4-340a5574b3c9/pull/0.log"
Jul 09 12:49:55.603231 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:55.603205    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xf9np_14f9643e-3daa-478e-824f-2a59d7d86877/kuadrant-console-plugin/0.log"
Jul 09 12:49:55.956112 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:55.956034    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-v4cbc_7bc591db-6b3f-4d74-9567-90f8890c76c5/limitador/0.log"
Jul 09 12:49:56.421688 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:56.421659    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sgxjr_493c258a-fdf6-488c-9b7f-6a304b59b578/istio-proxy/0.log"
Jul 09 12:49:56.879952 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:56.879922    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t8nbr_df01513e-f269-4c17-8272-b80b59e1810a/istio-proxy/1.log"
Jul 09 12:49:57.779124 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:57.779099    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-vhgl4_6fff3852-dc80-40bf-9732-90f525bc0729/main/0.log"
Jul 09 12:49:57.996929 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:57.996897    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-hzhtk_4685e84e-edb6-4ea1-b396-db534f083bcd/main/0.log"
Jul 09 12:49:58.105948 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:49:58.105918    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-6q4kg_9255360e-c0fe-48a2-b920-421af01ceccc/main/0.log"
Jul 09 12:50:05.606755 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:05.606659    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p456j_e3382b7a-7abc-4a2c-b06c-c1e8f7d1310b/global-pull-secret-syncer/0.log"
Jul 09 12:50:05.693922 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:05.693890    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-q4n45_d007f736-c0e5-4cf1-a25c-2dd14dbacb72/konnectivity-agent/0.log"
Jul 09 12:50:05.816714 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:05.816659    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-94.ec2.internal_ed9a03f52aaae8a3339d4a55bea80f61/haproxy/0.log"
Jul 09 12:50:10.396416 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:10.396380    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl_7cd144f5-bf25-4752-8353-3c549037a44a/extract/0.log"
Jul 09 12:50:10.429413 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:10.429380    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl_7cd144f5-bf25-4752-8353-3c549037a44a/util/0.log"
Jul 09 12:50:10.451126 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:10.451093    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759j6dwl_7cd144f5-bf25-4752-8353-3c549037a44a/pull/0.log"
Jul 09 12:50:10.484321 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:10.484290    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5_6efa932f-daab-4dfa-98af-eb221866d49f/extract/0.log"
Jul 09 12:50:10.508450 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:10.508424    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5_6efa932f-daab-4dfa-98af-eb221866d49f/util/0.log"
Jul 09 12:50:10.536096 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:10.536067    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06nkq5_6efa932f-daab-4dfa-98af-eb221866d49f/pull/0.log"
Jul 09 12:50:10.566092 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:10.566054    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6_949368a2-d606-447f-83c9-e1a9cfb5798e/extract/0.log"
Jul 09 12:50:10.594240 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:10.594216    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6_949368a2-d606-447f-83c9-e1a9cfb5798e/util/0.log"
Jul 09 12:50:10.622059 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:10.622032    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvbf6_949368a2-d606-447f-83c9-e1a9cfb5798e/pull/0.log"
Jul 09 12:50:10.656184 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:10.656108    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw_9d0ebd1d-d802-492e-93d4-340a5574b3c9/extract/0.log"
Jul 09 12:50:10.682218 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:10.682190    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw_9d0ebd1d-d802-492e-93d4-340a5574b3c9/util/0.log"
Jul 09 12:50:10.707176 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:10.707147    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef128qzw_9d0ebd1d-d802-492e-93d4-340a5574b3c9/pull/0.log"
Jul 09 12:50:11.170505 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:11.170477    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-xf9np_14f9643e-3daa-478e-824f-2a59d7d86877/kuadrant-console-plugin/0.log"
Jul 09 12:50:11.421363 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:11.421283    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-v4cbc_7bc591db-6b3f-4d74-9567-90f8890c76c5/limitador/0.log"
Jul 09 12:50:13.357911 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:13.357876    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xnp29_ba5787df-e1ac-4ae8-af04-901fc636f023/cluster-monitoring-operator/0.log"
Jul 09 12:50:13.397991 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:13.397963    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-bbrvx_b722a44f-8b60-4d9e-8a85-c699cb67f012/kube-state-metrics/0.log"
Jul 09 12:50:13.420142 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:13.420115    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-bbrvx_b722a44f-8b60-4d9e-8a85-c699cb67f012/kube-rbac-proxy-main/0.log"
Jul 09 12:50:13.449544 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:13.449514    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-bbrvx_b722a44f-8b60-4d9e-8a85-c699cb67f012/kube-rbac-proxy-self/0.log"
Jul 09 12:50:13.482512 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:13.482481    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5f76b8b7c6-gmhqp_81fab021-35cd-4aa1-abc6-be93fe07be1a/metrics-server/0.log"
Jul 09 12:50:13.759999 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:13.759918    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-slqgq_85aaac54-cd86-4b4c-a917-25ab0245c451/node-exporter/0.log"
Jul 09 12:50:13.803071 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:13.803041    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-slqgq_85aaac54-cd86-4b4c-a917-25ab0245c451/kube-rbac-proxy/0.log"
Jul 09 12:50:13.859987 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:13.859962    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-slqgq_85aaac54-cd86-4b4c-a917-25ab0245c451/init-textfile/0.log"
Jul 09 12:50:13.910877 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:13.910843    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-s75s6_7e7f8647-20da-49b4-94b8-d55124ff3b94/kube-rbac-proxy-main/0.log"
Jul 09 12:50:13.934282 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:13.934257    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-s75s6_7e7f8647-20da-49b4-94b8-d55124ff3b94/kube-rbac-proxy-self/0.log"
Jul 09 12:50:13.958430 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:13.958407    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-s75s6_7e7f8647-20da-49b4-94b8-d55124ff3b94/openshift-state-metrics/0.log"
Jul 09 12:50:14.163970 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:14.163939    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-vjp6w_a88fb4f2-69ae-4746-ba6a-d38ca7e46986/prometheus-operator/0.log"
Jul 09 12:50:14.185119 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:14.185095    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-vjp6w_a88fb4f2-69ae-4746-ba6a-d38ca7e46986/kube-rbac-proxy/0.log"
Jul 09 12:50:15.658213 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.658170    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"]
Jul 09 12:50:15.658748 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.658729    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec3b98ba-a4f9-4b32-bc7b-4798890c4443" containerName="maas-api"
Jul 09 12:50:15.658860 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.658751    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec3b98ba-a4f9-4b32-bc7b-4798890c4443" containerName="maas-api"
Jul 09 12:50:15.658860 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.658778    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7294ccdc-2d00-47fc-8dc0-42f7a8c304df" containerName="maas-api"
Jul 09 12:50:15.658860 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.658787    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7294ccdc-2d00-47fc-8dc0-42f7a8c304df" containerName="maas-api"
Jul 09 12:50:15.659018 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.658903    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7294ccdc-2d00-47fc-8dc0-42f7a8c304df" containerName="maas-api"
Jul 09 12:50:15.659018 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.658917    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec3b98ba-a4f9-4b32-bc7b-4798890c4443" containerName="maas-api"
Jul 09 12:50:15.663434 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.663406    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.666549 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.666524    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4jt4g\"/\"kube-root-ca.crt\""
Jul 09 12:50:15.666719 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.666550    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4jt4g\"/\"default-dockercfg-n6mxt\""
Jul 09 12:50:15.666719 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.666578    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4jt4g\"/\"openshift-service-ca.crt\""
Jul 09 12:50:15.668723 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.668684    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"]
Jul 09 12:50:15.782205 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.782167    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b69b697-b520-46da-b797-158d61ea19d9-podres\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.782404 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.782258    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z68q\" (UniqueName: \"kubernetes.io/projected/6b69b697-b520-46da-b797-158d61ea19d9-kube-api-access-6z68q\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.782404 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.782291    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b69b697-b520-46da-b797-158d61ea19d9-lib-modules\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.782404 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.782311    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b69b697-b520-46da-b797-158d61ea19d9-proc\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.782404 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.782386    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b69b697-b520-46da-b797-158d61ea19d9-sys\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.883591 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.883544    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z68q\" (UniqueName: \"kubernetes.io/projected/6b69b697-b520-46da-b797-158d61ea19d9-kube-api-access-6z68q\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.883807 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.883610    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b69b697-b520-46da-b797-158d61ea19d9-lib-modules\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.883807 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.883642    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b69b697-b520-46da-b797-158d61ea19d9-proc\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.883807 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.883677    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b69b697-b520-46da-b797-158d61ea19d9-sys\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.883807 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.883778    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b69b697-b520-46da-b797-158d61ea19d9-sys\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.883807 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.883782    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6b69b697-b520-46da-b797-158d61ea19d9-proc\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.884043 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.883828    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b69b697-b520-46da-b797-158d61ea19d9-lib-modules\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.884043 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.883838    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b69b697-b520-46da-b797-158d61ea19d9-podres\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.884043 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.883902    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6b69b697-b520-46da-b797-158d61ea19d9-podres\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.892121 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.892095    2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z68q\" (UniqueName: \"kubernetes.io/projected/6b69b697-b520-46da-b797-158d61ea19d9-kube-api-access-6z68q\") pod \"perf-node-gather-daemonset-h4kpk\" (UID: \"6b69b697-b520-46da-b797-158d61ea19d9\") " pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:15.981322 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:15.981225    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:16.126049 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:16.126020    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"]
Jul 09 12:50:16.128970 ip-10-0-141-94 kubenswrapper[2565]: W0709 12:50:16.128944    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6b69b697_b520_46da_b797_158d61ea19d9.slice/crio-0dba7f830bf90abdf17d35013fb10b8d57156b67c7471d09bfeee5cd5b3d47db WatchSource:0}: Error finding container 0dba7f830bf90abdf17d35013fb10b8d57156b67c7471d09bfeee5cd5b3d47db: Status 404 returned error can't find the container with id 0dba7f830bf90abdf17d35013fb10b8d57156b67c7471d09bfeee5cd5b3d47db
Jul 09 12:50:16.168536 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:16.168494    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk" event={"ID":"6b69b697-b520-46da-b797-158d61ea19d9","Type":"ContainerStarted","Data":"0dba7f830bf90abdf17d35013fb10b8d57156b67c7471d09bfeee5cd5b3d47db"}
Jul 09 12:50:17.175452 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:17.175416    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk" event={"ID":"6b69b697-b520-46da-b797-158d61ea19d9","Type":"ContainerStarted","Data":"a7724e79e8d4c5351ac2b8d8b1830ce5a5bb63ffcfd3df58d96383215cf0dee2"}
Jul 09 12:50:17.175891 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:17.175503    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:17.192854 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:17.192800    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk" podStartSLOduration=2.192783806 podStartE2EDuration="2.192783806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 12:50:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 12:50:17.1903879 +0000 UTC m=+2889.867916959" watchObservedRunningTime="2026-07-09 12:50:17.192783806 +0000 UTC m=+2889.870312864"
Jul 09 12:50:17.413382 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:17.413353    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-xl7np_d4216769-1d00-4c9b-b986-a577ad840711/volume-data-source-validator/0.log"
Jul 09 12:50:18.534449 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:18.534422    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xw2dj_f77b9023-6512-43a6-8d89-2ea512bc8c10/dns/0.log"
Jul 09 12:50:18.556973 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:18.556943    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xw2dj_f77b9023-6512-43a6-8d89-2ea512bc8c10/kube-rbac-proxy/0.log"
Jul 09 12:50:18.633240 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:18.633210    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zjcb9_760d11e4-c839-41a2-8e35-b4d2566ef960/dns-node-resolver/0.log"
Jul 09 12:50:19.244067 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:19.244037    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-r5k4m_8ce74bea-4ad5-4a3e-89a3-a04cab4b8ae2/node-ca/0.log"
Jul 09 12:50:20.198927 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:20.198896    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sgxjr_493c258a-fdf6-488c-9b7f-6a304b59b578/istio-proxy/0.log"
Jul 09 12:50:20.374848 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:20.374818    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t8nbr_df01513e-f269-4c17-8272-b80b59e1810a/istio-proxy/0.log"
Jul 09 12:50:20.401484 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:20.401446    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t8nbr_df01513e-f269-4c17-8272-b80b59e1810a/istio-proxy/1.log"
Jul 09 12:50:21.206842 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:21.206808    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dcmlh_f68292df-cc6d-4528-bac8-3a6badb20ed2/serve-healthcheck-canary/0.log"
Jul 09 12:50:21.734526 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:21.734493    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-k662b_3b9d000a-c712-4679-8044-c0a4f18920f0/insights-operator/0.log"
Jul 09 12:50:21.735454 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:21.735429    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-k662b_3b9d000a-c712-4679-8044-c0a4f18920f0/insights-operator/1.log"
Jul 09 12:50:21.758384 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:21.758355    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8rrb2_2b9c3ac6-03da-448e-b885-4e7edee28684/kube-rbac-proxy/0.log"
Jul 09 12:50:21.785969 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:21.785941    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8rrb2_2b9c3ac6-03da-448e-b885-4e7edee28684/exporter/0.log"
Jul 09 12:50:21.812413 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:21.812388    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8rrb2_2b9c3ac6-03da-448e-b885-4e7edee28684/extractor/0.log"
Jul 09 12:50:23.191235 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:23.191206    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4jt4g/perf-node-gather-daemonset-h4kpk"
Jul 09 12:50:24.136082 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:24.136048    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jbkjg_4e58dcac-ffcf-4e3f-a36f-83591970b8c4/manager/0.log"
Jul 09 12:50:24.606973 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:24.606940    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd7989cf5-5d2r2_4a0bbd67-ef5e-47c1-8f29-cde9280de159/manager/0.log"
Jul 09 12:50:24.627903 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:24.627876    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bd7989cf5-5d2r2_4a0bbd67-ef5e-47c1-8f29-cde9280de159/copy-manifests/0.log"
Jul 09 12:50:26.054497 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:26.054469    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-849fbb9d-frqzs_31e5831b-47b2-49b4-8a8c-b93f730edc3b/manager/0.log"
Jul 09 12:50:26.280429 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:26.280397    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sgxjr_493c258a-fdf6-488c-9b7f-6a304b59b578/istio-proxy/0.log"
Jul 09 12:50:26.416215 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:26.416179    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t8nbr_df01513e-f269-4c17-8272-b80b59e1810a/istio-proxy/0.log"
Jul 09 12:50:26.446200 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:26.446160    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t8nbr_df01513e-f269-4c17-8272-b80b59e1810a/istio-proxy/1.log"
Jul 09 12:50:27.752195 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:27.752157    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4sgxjr_493c258a-fdf6-488c-9b7f-6a304b59b578/istio-proxy/0.log"
Jul 09 12:50:27.890942 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:27.890904    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t8nbr_df01513e-f269-4c17-8272-b80b59e1810a/istio-proxy/0.log"
Jul 09 12:50:27.920008 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:27.919975    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t8nbr_df01513e-f269-4c17-8272-b80b59e1810a/istio-proxy/1.log"
Jul 09 12:50:32.880857 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:32.880827    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qf2v9_b994ce8a-e5c9-4828-b792-c9322dd75c26/kube-multus-additional-cni-plugins/0.log"
Jul 09 12:50:32.903189 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:32.903160    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qf2v9_b994ce8a-e5c9-4828-b792-c9322dd75c26/egress-router-binary-copy/0.log"
Jul 09 12:50:32.924364 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:32.924334    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qf2v9_b994ce8a-e5c9-4828-b792-c9322dd75c26/cni-plugins/0.log"
Jul 09 12:50:32.945831 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:32.945803    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qf2v9_b994ce8a-e5c9-4828-b792-c9322dd75c26/bond-cni-plugin/0.log"
Jul 09 12:50:32.967186 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:32.967151    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qf2v9_b994ce8a-e5c9-4828-b792-c9322dd75c26/routeoverride-cni/0.log"
Jul 09 12:50:32.989919 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:32.989891    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qf2v9_b994ce8a-e5c9-4828-b792-c9322dd75c26/whereabouts-cni-bincopy/0.log"
Jul 09 12:50:33.011530 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:33.011499    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qf2v9_b994ce8a-e5c9-4828-b792-c9322dd75c26/whereabouts-cni/0.log"
Jul 09 12:50:33.047279 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:33.047250    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mn4b9_b8802ef7-f67c-4aee-81cd-858b426464d9/kube-multus/0.log"
Jul 09 12:50:33.243616 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:33.243534    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rltxg_4ac6e98e-e488-4741-aa67-2a6a130291d8/network-metrics-daemon/0.log"
Jul 09 12:50:33.267455 ip-10-0-141-94 kubenswrapper[2565]: I0709 12:50:33.267431    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rltxg_4ac6e98e-e488-4741-aa67-2a6a130291d8/kube-rbac-proxy/0.log"